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Introduction 


For the last twenty-five years I have written a series of engineering mathematics books. 
When it came to revising my fourth edition, I realized that something radical must be 
done. The encyclopedic engineering mathematics tome is dead, killed by the growth of 
the Internet and students’ unwillingness to buy these books. When I surveyed the current 
variety of engineering mathematics courses, I realized that I needed to write two books. 
The companion to the present volume (Advanced Engineering Mathematics with MATLAB) 
would only focus on those topics that are currently taught in most advanced engineering 
mathematics courses. When one had studied that volume, he/she could feel confident that 
he/she had a solid knowledge of those mathematical techniques used in current engineering 
and scientific courses. 

This volume (Advanced Engineering Mathematics: A Second Course) is my attempt to 
look into the future of advanced engineering mathematics courses. Some of the material, 
such as complex variables and probability, is currently taught to engineers, although not 
usually in courses entitled advanced engineering mathematics. I have included these topics 
because they are required for transform methods and random processes. 

One trend that I see is that entering freshmen are increasingly likely to have had 
calculus in high school. This means that they will probably place out of the traditional 
differential and integral calculus courses in their freshman year, allowing them to take 
multivariable calculus and differential equations during their freshman year and taking 
advanced engineering mathematics courses during their sophomore year. Therefore, the 
question arises as to nature of these courses. The answer appears to be that the current 
traditional engineering mathematics course will occur during the fall semester and some 
other mathematics course will occur during the spring semester. The present volume is 
designed to meet this need, as well as stand as the advanced engineering mathematics 
text on its own. For those past formal education, this book provides the professional with 
powerful mathematical techniques. 

The first five chapters are aimed at the systems, communications and electrical en- 
gineering crowd: those involved in the digital revolution. First, that portion of complex 
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variable theory is presented so that the reader will feel prepared in dealing with transform 
methods. For example, Chapter 2 shows how complex variables can be used to invert par- 
ticularly complicated Fourier and Laplace transforms. This chapter also illustrates how 
transform methods can solve the heat, wave and Laplace’s equations. 

In Chapters 3 and 4 we study two transforms, the z- and Hilbert transforms, that are 
currently important in the digital revolution. Chapter 3 introduces the z-transform by first 
giving its definition and then developing some of its general properties. We also illustrate 
how to compute the inverse by long division, partial fractions, and contour integration. 
Finally, we use z-transforms to solve difference equations, especially with respect to the 
stability of the system. 

The Hilbert transform is important in the explosion of interest in communications. The 
Hilbert transform is introduced in Section 4.1 and its properties are explored in Section 4.2. 
Two important applications of Hilbert transforms are introduced in Sections 4.3 and 4.4, 
namely the concept of analytic signals and the Kramers-Kronig relationship. 

To round out this area we present Green’s function in Chapter 5. Green’s function 
gives the response of a system to impulse forcing without the clouding effects of a particular 
forcing function or initial conditions. Each successive section deals with ordinary, wave, heat 
and Helmholtz’s equations. The solution to general problems follows from the superposition 
integral. 

The book concludes by turning to the future. It is now recognized that random pro- 
cesses are useful in describing many physical systems. We begin by introducing the fun- 
damental concepts behind probability in Chapter 6 and random processes in Chapter 7. 
Chapter 6 introduces the student to the concepts of probability distributions, mean, and 
variance because these topics appear so frequently in random processes. Chapter 7 explores 
common random processes such as Poisson processes and birth and death. 

A unique aspect of this book appears in Chapter 8, which is devoted to stochastic 
calculus. We start by exploring deterministic differential equations with a stochastic forcing. 
Next, the important stochastic process of Brownian motion is developed in depth. Using this 
Brownian motion, we introduce the concept of (It6) stochastic integration, Itô’s lemma, and 
stochastic differential equations. The chapter concludes with various numerical methods to 
integrate stochastic differential equations. 

MATLAB is still employed to reinforce the concepts that are taught. Of course, this 
book still continues my principle of including a wealth of examples from the scientific and 
engineering literature. Worked solutions to all of the problems are given at the end. 


List of Definitions 


Function Definition 
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Chapter 1 


Complex Variables 


The theory of complex variables was originally developed by mathematicians as an aid 
in understanding functions. Functions of a complex variable enjoy many powerful properties 
that their real counterparts do not. That is not why we will study them. For us they provide 
the keys for the complete mastery of transform methods and differential equations. 

In this chapter all of our work points to one objective: integration on the complex 
plane by the method of residues. For this reason we minimize discussions of limits and 
continuity, which play such an important role in conventional complex variables, in favor 
of the computational aspects. We begin by introducing some simple facts about complex 
variables. Then we progress to differential and integral calculus on the complex plane. 


1.1 COMPLEX NUMBERS 


A complex number is any number of the form a+bi, where a and b are real andi = y—1. 
We denote any member of a set of complex numbers by the complex variable z = x + ty. 
The real part of z, usually denoted by R(z), is x while the imaginary part of z, S(z), is y. 
The complex conjugate, Z or z*, of the complex number a + bi is a — bi. 

Complex numbers obey the fundamental rules of algebra. Thus, two complex numbers 
a + bi and c+ di are equal if and only if a = c and b = d. Just as real numbers have 
the fundamental operations of addition, subtraction, multiplication, and division, so too do 
complex numbers. These operations are defined: 


Addition 
(a+ bi) + (c+ di) = (a+c)+(b+d)i (1.1.1) 


Subtraction 


(a + bi) — (c + di) = (a — c) + (b — d)i (1.1.2) 
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Multiplication 
(a + bi)(c + di) = ac + bci + adi + i7bd = (ac — bd) + (ad + be)i (1.1.3) 
Division 
a+bi at+bic—di  ac—adi+bci—bdi? ac+bd+ (be—ad)i 


= 2 Z . 1.1.4 
c+di c+dic-—di c? + d? c2 + d2 ( ) 


The absolute value or modulus of a complex number a + bi, written |a + bi|, equals 
Va? + b?. Additional properties include: 


|212223°++ Zn] = |z1||z2l|23|--- |en| (1.1.5) 
|z1/z| = |z1|/|z2| if 22 x 0 (1.1.6) 
lz + za + 23 +: + 2n| < [zil + [ze] + lea] +--+ [znl (1.1.7) 
and 


The use of inequalities with complex variables has meaning only when they involve absolute 
values. 

It is often useful to plot the complex number x + iy as a point (x,y) in the xy-plane, 
now called the complex plane. Figure 1.1.1 illustrates this representation. 

This geometrical interpretation of a complex number suggests an alternative method 
of expressing a complex number: the polar form. From the polar representation of x and y, 


x=rcos(?) and y=rsin(@), (1.1.9) 


where r = \/x? + y? is the modulus, amplitude, or absolute value of z and @ is the argument 
or phase, we have that 
z = x + iy = r|cos(0) + isin(0)]. (1.1.10) 


However, from the Taylor expansion of the exponential in the real case, 


a $ (0p 

gee ys ei ; (1.1.11) 
k=0 
Expanding Equation 1.1.11, 
02 o o 6 E o 
i0 = fi | | 4 | | 

er = 2l T 4! 6! T Tî ( 3l T 5I 7! T ) (1.1.12) 
= cos(0) + i sin(0). (1.1.13) 


Equation 1.1.13 is Fuler’s formula. Consequently, we may express Equation 1.1.10 as 
z=re’®, (1.1.14) 
which is the polar form of a complex number. Furthermore, because 


Zh ah e (1.1.15) 
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Figure 1.1.1: The complex plane. 


by the law of exponents, 
z” = r” [cos(n0) + isin(né)]. (1.1.16) 


Equation 1.1.16 is De Moivre’s theorem. 
e Example 1.1.1 


Let us simplify the following complex number: 


3-2 3-2 1-i 3-—31+214277 -5-i 5 i 
= = - = (11.1 
z Shi e I}1 2 po ET 


e Example 1.1.2 


Let us reexpress the complex number — v6 — iV/2 in polar form. From Equation 1.1.9 
r= /6+2 and 0 = tan7!(b/a) = tan7!(1/V3) = 7/6 or 77/6. Because — v6 — iv/2 lies in 
the third quadrant of the complex plane, 0 = 77/6 and 


— V6 — iV2 = 2 2077/6, (1.1.18) 


Note that Equation 1.1.18 is not a unique representation because +2na may be added to 
77/6 and we still have the same complex number since 


i(O+2n7) 


e = cos(6 + 2nr) + isin(0 + 2nr) = cos(0) + isin(0) = e”. (1.1.19) 


For uniqueness we often choose n = 0 and define this choice as the principal branch. Other 
branches correspond to different values of n. 


e Example 1.1.3 


Find the curve described by the equation |z — zo| = a. 
From the definition of the absolute value, 


V(x — 20)? + (y — yo)? =a (1.1.20) 
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(x — z0)? + (y— yo)? =a’. (1.1.21) 


Equation 1.1.21, and hence |z — zo| = a, describes a circle of radius a with its center located 
at (£o, Yo). Later on, we shall use equations such as this to describe curves in the complex 
plane. 


e Example 1.1.4 
As an example in manipulating complex numbers, let us show that 


a+ bi 


= 1.1.22 
b+at ( ) 
We begin by simplifying 

a+bi a+bi b-ai 2ab b? — a? 
= = } }. 1.1.23 
b+ai bra b—ai 2+ aie ( ) 

Therefore, 
bi 4a2b2 bt — 2a2b2 + q4 4 4 2q2b2 + b4 

Grk g R cl a AEN | lee (1.1.24) 

b+ at (a2 + b2)2 (a2 + b2)2 (a2 + b2)2 


MATLAB can also be used to solve this problem. Typing the commands 


>> syms a b real 
>> abs((atb*i)/(bta*i)) 


yields 


ans = 
1 
Note that you must declare a and b real in order to get the final result. 


Problems 


Simplify the following complex numbers. Represent the solution in the Cartesian form a+bi. 
Check your answers using MATLAB. 


5i 5+5i 20 l1+2i 2—i 
'2+i “3-44 4+3i '3—4i 5i 

. ; ; . 7+i)(1-— 5i) 

4. (1 —1)# 5. i(1 — iv 3)(v3 6. UPOLE 

a-i) i(1 = iv3)(V3 + i) E 

Represent the following complex numbers in polar form: 

7. —i 8. —4 9. 24273: 
10. —5+ 5% 11. 2—2i 12. —1+v3i 


13. By the law of exponents, et(%+t2) = e’%e’®, Use Euler’s formula to obtain expressions 
for cos(a + 8) and sin(a + 8) in terms of sines and cosines of a and £. 
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14. Use De Moivre’s theorem with r = 1 to express cos(40) and sin(4@) in terms of cos(@) 
and sin(@). 


15. Using the property that pS q = (1 — qXt*)/(1 — q) and the geometric series 
Di et”t, obtain the following sums of trigonometric functions: 


ec Nt\ sin[(N + 1)t/2] eae _ (Nt) sin[(N + 1)t/2] 
5 cos(nt) = cos( 5 ) sin(t/2) and 5 sin(nt) = sin( 7 ) sin(t/2) 


n=0 n=1 


These results are often called Lagrange’s trigonometric identities. 


16. (a) Using the property that Dog” = 1/(1 — q), if || < 1, and the geometric series 
rg ere’, |e] < 1, show that 


n= 


e sin(t) 
1 +€ — 2e cos(t)” 


S 1 — e cos(t) ES gh 
2 cos(nt) = Tope = de Cost) and 2: sin(nt) = 


(b) Let e = e~*, where a > 0. Show that 


sin(t) 
cosh(a) — cos(t) ` 


2 5 e"@ sin(nt) = 
n=1 


1.2 FINDING ROOTS 


The concept of finding roots of a number, which is rather straightforward in the case of 
real numbers, becomes more difficult in the case of complex numbers. By finding the roots 
of a complex number, we wish to find all of the solutions w of the equation w” = z, where 
n is a positive integer for a given z. 

We begin by writing z in the polar form: 


gore, (1.2.1) 
while we write l 
w = Ret? (1.2.2) 
for the unknown. Consequently, 
w = R"e”? = re? = z, (1.2.3) 
We satisfy Equation 1.2.3 if 
R” =r and n®=y+2kn, k=0,+1,+2,..., (1.2.4) 


because the addition of any multiple of 27 to the argument is also a solution. Thus, 
R=r'/", where R is the uniquely determined real positive root, and 


ark 
Cpe es ESEL aa (1.2.5) 
n n 
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Because wy, = Wk+n, it is sufficient to take k = 0,1,2,...,n—1. Therefore, there are exactly 
n solutions: > 
; 2 
wre = Re?* = r!/” exp f € + =*)| (1.2.6) 
n n 
with k = 0,1,2,...,n — 1. They are the n roots of z. Geometrically we can locate these 


solutions w% on a circle, centered at the point (0,0), with radius R and separated from each 
other by 27/n radians. These roots also form the vertices of a regular polygon of n sides 
inscribed inside a circle of radius R. (See Example 1.2.1.) 

In summary, the method for finding the n roots of a complex number Zo is as follows. 
First, write zp in its polar form: z9 = re’*?. Then multiply the polar form by e?’**. Using 
the law of exponents, take the 1/n power of both sides of the equation. Finally, using 
Euler’s formula, evaluate the roots for k = 0,1,...,n—1. 


e Example 1.2.1 


Let us find all of the values of z for which z° = —32 and locate these values on the 
complex plane. 
Because 
—$2 = 9267" = Pe, (1.2.7) 
| 2rik 
= 2e(F + T iF k= 0,1,2,3,4, (1.2.8) 
or 
zo = 2exr( =) =2 [cos (=) + isin(=)| ; (1.2.9) 
Biss Sp Neca 2) hae (| (1.2.10) 
5 5 5 
z = 2e™ = —2, (1.2.11) 
$e) Oy Fee) ee (1.2.12) 
5 5 5 
and 


sra =afom(%)+iin(%)]. a 


Figure 1.2.1 shows the location of these roots in the complex plane. 


e Example 1.2.2 


Let us find the cube roots of —1 + ¿i and locate them graphically. 
Because —1 + i = V2 exp(37i/4), 


| 2ink 
= 2 exp( 3 + = 


) , k=0,1,2, (1.2.14) 


or 


zo = 21/6 ep( Z) = 21/6 [cos(F) +isin(Z)] , (1.2.15) 
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= 
X 
Figure 1.2.1: The zeros of 2° = —32. 
lliri llr lir 
_ 91/6 — 91/6 : me 
z =2 ex( E ) 2 feos( =) isin( T )). (1.2.16) 
an 1971 19 9 
_ 91/6, mi _ 91/6 T aE 197 
zo = 2 ep ) 2 feos( 5 isin| -5 ; (1.2.17) 


Figure 1.2.2 gives the location of these zeros on the complex plane. 
e Example 1.2.3 


The routine solve in MATLAB can also be used to compute the roots of complex 
numbers. For example, let us find all of the roots of z4 = —a*. 


The MATLAB commands are as follows: 
>> syms a Z 
>> solve(z^4+a^4) 
This yields the solution 
ans= 
[ (4/2*2^(1/2)+1/2xi*2^(1/2))*a] 
[ (-1/2*2^(1/2)+1/2*xi*2^(1/2))*a] 
[ (4/2*2^(1/2)-1/2xi*2^(1/2))*a] 
[ (-1/2*2^(1/2)-1/2*xi*2^(1/2))*a] 


Problems 


Extract all of the possible roots of the following complex numbers. Verify your answer using 
MATLAB. 


1. 81/6 2. (—=1)⁄3 3. (—i)1/3 4. (—27)1/8 
5. Find algebraic expressions for the square roots of a — bi, where a > 0 and b > 0. 


6. Find all of the roots for the algebraic equation 24 — 3iz? — 2 = 0. Then check your answer 
using solve in MATLAB. 
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Figure 1.2.2: The zeros of z3 = —1 + i. 


7. Find all of the roots for the algebraic equation z+ + 6iz? + 16 = 0. Then check your 
answer using solve in MATLAB. 


1.3 THE DERIVATIVE IN THE COMPLEX PLANE: THE CAUCHY-RIEMANN EQUATIONS 


In the previous two sections, we introduced complex arithmetic. We are now ready for 
the concept of function as it applies to complex variables. 

We already defined the complex variable z = x+iy, where x and y are variable. We now 
introduce another complex variable w = u+iv so that for each value of z there corresponds 
a value of w = f(z). From all of the possible complex functions that we might invent, we 
focus on those functions where for each z there is one, and only one, value of w. These 
functions are single-valued. They differ from functions such as the square root, logarithm, 
and inverse sine and cosine, where there are multiple answers for each z. These multivalued 
functions do arise in various problems. However, they are beyond the scope of this book 
and we shall always assume that we are dealing with single-valued functions. 

A popular method for representing a complex function involves drawing some closed 
domain in the z-plane and then showing the corresponding domain in the w-plane. This 
procedure is called mapping and the z-plane illustrates the domain of the function while 
the w-plane illustrates its image or range. Figure 1.3.1 shows the z-plane and w-plane for 
w = z*; a pie-shaped wedge in the z-plane maps into a semicircle on the w-plane. 


e Example 1.3.1 


zZ 


Given the complex function w = e7 a let us find the corresponding u(x, y) and v(x, y). 


From Euler’s formula, 
wae = eT EtU)? = ya 9 2iey — ya? [cos(2xy) — isin(2zxy)]. (1.3.1) 


Therefore, by inspection, 


u(x, y) = eve cos(2xy), and v(x, y) = ae" sin(2xy). (1.3.2) 
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Figure 1.3.1: The complex function w = z?. 


Note that there is no i in the expression for u(x, y). The function w = f(z) is single-valued 
because for each distinct value of z, there is a unique value of u(x, y) and v(x, y). 


e Example 1.3.2 


As counterpoint, let us show that w = /z is a multivalued function. 
We begin by writing z = ret? t2rik where r = \/a? + y? and 0 = tan~!(y/x). Then, 


wp = yre?/ trik, k=0,1, (1.3.3) 
or 
wo = vyr [cos(0/2) + isin(0/2)) and wi =—wo. (1.3.4) 
Therefore, 
uo(x, y) = yr cos(0/2), vo(x, y) = vr sin(0/2), (1.3.5) 
and 
u(x, y) = — vyr cos(0/2), v(x, y) = — vyr sin(0/2). (1.3.6) 


Each solution wo or w is a branch of the multivalued function yz. We can make yz single- 
valued by restricting ourselves to a single branch, say wo. In that case, the R(w) > 0 if we 
restrict —7 < 0 < m. Although this is not the only choice that we could have made, it is a 
popular one. For example, most digital computers use this definition in their complex square 
root function. The point here is our ability to make a multivalued function single-valued 
by defining a particular branch. 


Although the requirement that a complex function be single-valued is important, it is 
still too general and would cover all functions of two real variables. To have a useful theory, 
we must introduce additional constraints. Because an important property associated with 
most functions is the ability to take their derivative, let us examine the derivative in the 
complex plane. 

Following the definition of a derivative for a single real variable, the derivative of a 
complex function w = f(z) is defined as 


dw _ li Aw _ ji f(z + 4z) -— f(z) 
dz Resp Az A220 Az j 


(1.3.7) 


A function of a complex variable that has a derivative at every point within a region of the 
complex plane is said to be analytic (or regular or holomorphic) over that region. If the 
function is analytic everywhere in the complex plane, it is entire. 
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Because the derivative is defined as a limit and limits are well behaved with respect 
to elementary algebraic operations, the following operations carry over from elementary 
calculus: 


£ lero] =cf'(z), c a constant (1.3.8) 
fe + o(2| = f'(z)+q'(z) (1.3.9) 
Eloo] -root roro (1.3.10) 

d | f(z) IOF) -sF 
dz Fale 7 Fe) ox? 
= {slo} = f'[g(z)|9’(z), the chain rule. (1.3.12) 


Another important property that carries over from real variables is ’H6pital’s rule: Let 
f(z) and g(z) be analytic at zo, where f(z) has a zero! of order m and g(z) has a zero of 
order n. Then, if m >n, 


lim f(z) = 0; (1.3.13) 
220 g(z) 
ifm=n, 
im TO = 10), (1.3.14) 


z>zo g(z) gl) (z0) 
and if m <n, 
ii l o: (1.3.15) 


e Example 1.3.3 


Let us evaluate lim,_,;(2'° + 1)/(z® + 1). From lHôpital’s rule, 


10 1 1 9 
lim ea im ere sae E = (1.3.16) 
Zt zê + 1 Zt 625 3 Zt 3 


So far, we introduced the derivative and some of its properties. But how do we actually 
know whether a function is analytic or how do we compute its derivative? At this point we 
must develop some relationships involving the known quantities u(x, y) and v(x, y). 

We begin by returning to the definition of the derivative. Because Az = Ax + iAy, 
there is an infinite number of different ways of approaching the limit Az > 0. Uniqueness 
of that limit requires that Equation 1.3.7 must be independent of the manner in which Az 
approaches zero. A simple example is to take Az in the z-direction so that Az = Ag; 
another is to take Az in the y-direction so that Az = iAy. These examples yield 


dw Aw Au+iAv ôu dv 

— = lm —e= li = + 4 1.3.17 

dz Az+0 Az  Ar>0 Aa dx | Ox ( ) 
1 An analytic function f(z) has a zero of order m at zo if and only if f (z0) = f’ (z0) = --- = f("~) (zo) = 


0 and f™ (zo) £0. 
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Although educated as an engineer, Augustin-Louis Cauchy (1789-1857) would become a mathe- 
matician’s mathematician, publishing 789 papers and 7 books in the fields of pure and applied 
mathematics. His greatest writings established the discipline of mathematical analysis as he refined 
the notions of limit, continuity, function, and convergence. It was this work on analysis that led him 
to develop complex function theory via the concept of residues. (Portrait courtesy of the Archives 
de l’Académie des sciences, Paris.) 


and 


dw E = lim An a = On ee (1.3.18) 


In both cases we are approaching zero from the positive side. For the limit to be unique 
and independent of path, Equation 1.3.17 must equal Equation 1.3.18, or 


Ou _ dv ðu 2 “08 (1.3.19) 
Ox Oy Oy ðr 


These equations that u and v must both satisfy are the Cauchy-Riemann equations. 
They are necessary but not sufficient to ensure that a function is differentiable. The follow- 
ing example illustrates this. 


e Example 1.3.4 


Consider the complex function 


Pyles z#0 
= y 1.3.2 
= { 0, z=0. a2) 
The derivative at z = 0 is given by 
5 4 4 
OO t i) SAO) OE ag A (1.3.21) 


dz Az30 Az ~ Az0 |Az|4’ 
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Despite his short life, (Georg Friedrich) Bernhard Riemann’s (1826-1866) mathematical work con- 
tained many imaginative and profound concepts. It was in his doctoral thesis on complex function 
theory (1851) that he introduced the Cauchy-Riemann differential equations. Riemann’s later work 
dealt with the definition of the integral and the foundations of geometry and non-Euclidean (elliptic) 
geometry. (Portrait courtesy of Photo AKG, London, with permission.) 


provided that this limit exists. However, this limit does not exist because, in general, the 
numerator depends upon the path used to approach zero. For example, if Az = re™*/4 with 
r — 0, dw/dz = —1. On the other hand, if Az = re™’/? with r > 0, dw/dz = 1. 

Are the Cauchy-Riemann equations satisfied in this case? To check this, we first com- 
pute 


Azx—->0 


4 i 4 
us(0,0) = lim (5) zis  », (0,0) = lim Ga =F, (1.3.22) 


: 5 
u,(0,0) = lim R| (iAy) |=0 and v,(0,0) = lim $ =0. (1.3.23) 


Ay>0 | Ay|Ay|* 


Hence, the Cauchy-Riemann equations are satisfied at the origin. Thus, even though the 
derivative is not uniquely defined, Equation 1.3.21 happens to have the same value for paths 
taken along the coordinate axes so that the Cauchy-Riemann equations are satisfied. 


In summary, if a function is differentiable at a point, the Cauchy-Riemann equations 
hold. Similarly, if the Cauchy-Riemann equations are not satisfied at a point, then the 
function is not differentiable at that point. This is one of the important uses of the Cauchy- 
Riemann equations: the location of nonanalytic points. Isolated nonanalytic points of an 
otherwise analytic function are called isolated singularities. Functions that contain isolated 
singularities are called meromorphic. 

The Cauchy-Riemann condition can be modified so that it is sufficient for the derivative 
to exist. Let us require that uz, Uy, vz, and vy be continuous in some region surrounding a 


Complex Variables 


point z and satisfy the Cauchy-Riemann equations there. Then 


f(z) — fo) = [u(z) — u(zo)] + tlv(z) — v(Z0)] 

= [ux (Z0)(@ — Zo) + Uy(Z0)(Y — Yo) + c1 (x — Zo) + €2(y — Yo) 
+ i[vz(20)(x — £o) + vy(20)(y — yo) + €3(% — Zo) + €a(y — Yo)] 
= [Ux (Zo) + ivz(z0)|(z — zo) j 
+ (e2 + i€4)(y — yo), 


13 


(1.3.24) 


(1.3.25) 


(1.3.26) 


where we used the Cauchy-Riemann equations and €1, €2, €3,€4 > 0 as Az, Ay > 0. Hence, 


am IO- F) 


1 = 
f (20) = Peay Az 


= Uz (20) + Wve(Z0), 


(1.3.27) 


because |Az| < |Az| and |Ay| < |Az|. Using Equation 1.3.27 and the Cauchy-Riemann 


equations, we can obtain the derivative from any of the following formulas: 


Ox Oy Oy’ 


and 


Furthermore, f’(zo) is continuous because the partial derivatives are. 
e Example 1.3.5 
Let us show that sin(z) is an entire function. 


w = sin(z) 
u + iv = sin(x + iy) = sin(x) cos(iy) + cos(x) sin(iy) 
= sin(x) cosh(y) + i cos(x) sinh(y), 


because 
cos(iy) = 4 [e%™) + ei)] = [et + e-¥] = cosh(y), 
2 2 
and 
sin(iy) = 4 [et — eW i) = —4 [e” — e™”] = isinh(y), 
so that 


u(x, y) = sin(x) cosh(y), and v(z,y)= cos(x)sinh(y). 


Differentiating both u(x, y) and v(x, y) with respect to x and y, we have that 
Ou 


Ou A A 
ae cos(x) cosh(y), a sin(x) sinh(y), 


(1.3.28) 


(1.3.29) 


(1.3.30) 
(1.3.31) 
(1.3.32) 


(1.3.33) 
(1.3.34) 


(1.3.35) 


(1.3.36) 
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a = — sin(x) sinh(y), oF = cos(x) cosh(y), (1.3.37) 


and u(x,y) and v(z,y) satisfy the Cauchy-Riemann equations for all values of x and y. 
Furthermore, uz, Uy, Vz, and vy are continuous for all x and y. Therefore, the function 
w = sin(z) is an entire function. 


e Example 1.3.6 


Consider the function w = 1/z. Then 


; 1 x iy 
= = = . 1.3.38 
w= u+ w ia Po Pey ( ) 
Therefore, 
x yY 
Now 
ðu (+y) -2 y-er (1.3.40) 
ðr (Fy Fy as 
ðv (a2 +y?)-2y? y- ðu daai 
ðy — (typ? — (æ? +y) ðr’ > 
Ov 0 — 2ay Qxy 
= a 1.3.42 
Oc +P as 
and 
ðu O0-2ry _ 2ry Ov (1.3.43) 


dy PP GPP Oe 


The function is analytic at all points except the origin because the function itself ceases to 
exist when both x and y are zero and the modulus of w becomes infinite. 


e Example 1.3.7 


Let us find the derivative of sin(z). 
Using Equation 1.3.28 and Equation 1.3.32, 


a sin(z)} = ae! = cos(x) cosh(y) —isin(z) sinh(y) = cos(a+iy) = cos(z). (1.3.44) 
dz Ox Ox 
Similarly, 
af ly\ y- r? l 2izy 1 a] 
dz (<) E (a? + y?)? i (a? + y?)? = (x + iy)? = Ja (1.3.45) 


The results in the above examples are identical to those for z real. As we showed 
earlier, the fundamental rules of elementary calculus apply to complex differentiation. Con- 
sequently, it is usually simpler to apply those rules to find the derivative rather than breaking 
f(z) down into its real and imaginary parts, applying either Equation 1.3.28 or Equation 
1.3.29, and then putting everything back together. 
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An additional property of analytic functions follows by cross differentiating the Cauchy- 
Riemann equations, or 


3u 3v 3u Ou Ou 
Ox? OxOy Oy?” Bx? x Oy2 9; a) 
a 0? 0? ə? ə? 0? 
v u v v v 


Any function that has continuous partial derivatives of second order and satisfies Laplace’s 
equation, Equation 1.3.46 or Equation 1.3.47, is called a harmonic function. Because both 
u(x, y) and v(x, y) satisfy Laplace’s equation if f(z) = u + iv is analytic, u(x, y) and v(x, y) 
are called conjugate harmonic functions. 


e Example 1.3.8 


Given that u(x, y) = e~*[xsin(y) — y cos(y)], let us show that u is harmonic and find 
a conjugate harmonic function v(x, y) such that f(z) = u + iv is analytic. 


Because 
o?u n ATI a 
a2 = —2e “ sin(y) + xe * sin(y) — ye ” cos(y), (1.3.48) 
and 
3u P7 ig E 
gyz = ze“ sin(y) + 2e ® sin(y) + ye” cos(y), (1.3.49) 
y 


it follows that Uss + Uyy = 0. Therefore, u(x, y) is harmonic. From the Cauchy-Riemann 
equations, 


ðv Ou 


Dy =z 7e sin(y) — xe” sin(y) + ye” cos(y), (1.3.50) 
and ə ə 
U U -r = —r _ -T e 
E e`” cos(y) — xe * cos(y) — ye “ sin(y). (1.3.51) 


Integrating Equation 1.3.50 with respect to y, 
u(x, y) = ye” sin(y) + xe~* cos(y) + g(x). (1.3.52) 
Using Equation 1.3.51, 


Uz = —ye—” sin(y) — xe~* cos(y) + e~” cos(y) + g'(x) 
=e *cos(y) — ze * cos(y) — ye” sin(x). (1.3.53 


Therefore, g'(x) = 0 or g(x) = constant. Consequently, 
v(x, y) =e *[ysin(y) + x cos(y)] + constant. (1.3.54) 


Hence, for our real harmonic function u(x, y), there are infinitely many harmonic conjugates 
v(a,y), which differ from each other by an additive constant. 
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Problems 


Show that the following functions are entire: 
1. f(z) =iz4+2 2. f(z) = e77 3. le) = 2 4. f(z) = cosh(z) 


Find the derivative of the following functions: 


5. f(z) = (1 + 22)8/ 6. f(z) = (z + 221/2)1/3 7. f(z) = (1+ 42)z? — 3z — 2 
8. f(z) = (2z —i)/(z+ 2i) 9. f(z) =(iz-1)73 10. f(z) = 2/(22 +1) 
Evaluate the following limits: 
a ae na E 
E eee i i 13. lim 
zi 24422241 230 z8 zon sin(7z) 


Here, n is an integer. 
14. Show that the function f(z) = z* is nowhere differentiable. 


For each of the following u(x, y), show that it is harmonic and then find a corresponding 
v(x, y) such that f(z) = u + iv is analytic. 


15. u(x, y) = 27 — y? 16. u(x, y) = zt — 6x?y? + yt +r 
17. u(x, y) = xcos(x)e~¥ — ysin(x)e™” 18. u(x, y) = (x? — y?) cos(y)e” — 2xy sin(y)e” 
1.4 LINE INTEGRALS 


So far, we discussed complex numbers, complex functions, and complex differentiation. 
We are now ready for integration. 

Just as we have integrals involving real variables, we can define an integral that involves 
complex variables. Because the z-plane is two-dimensional, there is clearly greater freedom 
in what we mean by a complex integral. For example, we might ask whether the integral of 
some function between points A and B depends upon the curve along which we integrate. 
(In general it does.) Consequently, an important ingredient in any complex integration is 
the contour that we follow during the integration. 

The result of a line integral is a complex number or expression. Unlike its counterpart 
in real variables, there is no physical interpretation for this quantity, such as area under 
a curve. Generally, integration in the complex plane is an intermediate process with a 
physically realizable quantity occurring only after we take its real or imaginary part. For 
example, in potential fluid flow, the lift and drag are found by taking the real and imaginary 
parts of a complex integral, respectively. 

How do we compute fo f(z) dz? Let us deal with the definition; we illustrate the actual 
method by examples. 

A popular method for evaluating complex line integrals consists of breaking everything 
up into real and imaginary parts. This reduces the integral to line integrals of real-valued 
functions, which we know how to handle. Thus, we write f(z) = u(x, y) +iv(x,y) as usual, 
and because z = x + iy, formally dz = dx + i dy. Therefore, 


1 f(z2)dz = 1 u(x, y) + iv(x,y)]|dxz + idy] (1.4.1) 
C C 


=| ue, y) de ~ v(z,u) dy +i f u(x, y) dx + u(x, y)dy. (1.4.2) 
c c 
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A y C 


Figure 1.4.1: Contour used in Example 1.4.1. 
The exact method used to evaluate Equation 1.4.2 depends upon the exact path specified. 


From the definition of the line integral, we have the following self-evident properties: 


f f(z\dz=— | f(z)dz, (1.4.3) 
C Cc’ 


where C” is the contour C taken in the opposite direction of C and 


J f(ie)dz= f(z) dz+ f(z) dz. (1.4.4) 
Ci+C2 C71 C2 


e Example 1.4.1 


Let us evaluate f cg 2 dz from z = 0 to z = 4+ 2i along two different contours. The first 
consists of the parametric equation z = t? + it. The second consists of two “dog legs”: the 
first leg runs along the imaginary axis from z = 0 to z = 2i and then along a line parallel 
to the z-axis from z = 2i to z = 4 + 2i. See Figure 1.4.1. 

For the first case, the points z = 0 and z = 4+ 27 on Cı correspond to t = 0 and t = 2, 
respectively. Then the line integral equals 


2 2 
| de = f (P + it)* d(t? + it) =} (2¢° — it? +t) dt = 10 — Ë. (1.4.5) 
Cy 0 0 


The line integral for the second contour C2 equals 


| ede= | azt | z*dz, (1.4.6) 
Co Coa Cap 


where Cp, denotes the integration from z = 0 to z = 27 while Co, denotes the integration 
from z = 2i to z = 4 + 2i. For the first integral, 


2 
/ a= f (x — iy)(dx + idy) -| y dy = 2, (1.4.7) 
Coa Cra 0 


because x = 0 and dz = 0 along C2a. On the other hand, along C2, y = 2 and dy = 0 so 
that 


4 4 
| de = f (x — iy)(dx + i dy) =| cde +i f —2 dr = 8 — 8i. (1.4.8) 
C20 Cov 0 0 
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Figure 1.4.2: Contour used in Example 1.4.2. 


Thus the value of the entire Ch contour integral equals the sum of the two parts, or 10 — 81. 

The point here is that integration along two different paths has given us different results 
even though we integrated from z = 0 to z = 4+ 2i both times. This result foreshadows 
a general result that is extremely important. Because the integrand contains nonanalytic 
points along and inside the region enclosed by our two curves, as shown by the Cauchy- 
Riemann equations, the results depend upon the path taken. Since complex integrations 
often involve integrands that have nonanalytic points, many line integrations depend upon 
the contour taken. 


e Example 1.4.2 


Let us integrate the entire function f(z) = z? along the two paths from z = 0 to 
z = 2 + i shown in Figure 1.4.2. For the first integration, x = 2y, while along the second 
path we have two straight paths: z = 0 to z = 2 and z = 2 to z = 2 +i. 

For the first contour integration, 


1 1 
J 2dz= f (2y + iy)” (2 dy + idy) = Í (3y? + 4y?2)(2 dy + idy) (1.4.9) 
Cı 0 0 


1 1 
=} Gy? dy + 8yPi dy + 3yPidy — Ay? ay = | 24? dy +11y°idy (1.4.10) 
0 0 


= 2y9 |p + Hig’ = 2 + HE. (1.4.11) 


For our second integration, 


f agal 2de+ | 2 dz. (1.4.12) 
C2 Cera Cav 


Along C2, we find that y = dy = 0 so that 


2 
J. z” dz -| ydr= ia h= (1.4.13) 
Coa 0 


1 3 
| z? d= f (2+ iy)idy =i (4y +21? - | 
Crp 0 3 


because x = 2 and dx = 0. Consequently, 


and 
t é 

=4i-2-ż, (1.4.14) 
0 


2 ll 


2 dz = Ź + —. 1.4.15 
f z=24 H (1.4.15) 
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Figure 1.4.3: Contour used in Example 1.4.3. 


In this problem we obtained the same results from two different contours of integration. 
Exploring other contours, we would find that the results are always the same; the integration 
is path independent. But what makes these results path independent while the integration 
in Example 1.4.1 was not? Perhaps it is the fact that the integrand is analytic everywhere 
on the complex plane and there are no nonanalytic points. We will explore this later. 


Finally, an important class of line integrals involves closed contours. We denote this 
special subclass of line integrals by placing a circle on the integral sign: ¢. Consider now 
the following examples: 


e Example 1.4.3 


Let us integrate f(z) = z around the closed contour shown in Figure 1.4.3. 


From Figure 1.4.3, 
psa f zaz+ | zaz+ | zdz. (1.4.16) 
C C1 C2 C3 


Now 
0 0 y? pe 
| zdz= f iy (idy) = - | ydy =- | =>, (1.4.17) 
C1 1 1 2 1 2 
= 1 Q)-1 
1 
i: zdz = ade=—| = =, (1.4.18) 
Co 0 2 lo 2 
and 
TJE o e28i 7/2 
/ zdz= f ete" dd = ——| =-1, (1.4.19) 
C3 -T 2 -r 


Oi 


where we used z = e”’ around the portion of the unit circle. Therefore, the closed line 


integral equals zero. 


e Example 1.4.4 


Let us integrate f(z) = 1/(z — a) around any circle centered on z = a. The Cauchy- 
Riemann equations show that f(z) is a meromorphic function. It is analytic everywhere 
except at the isolated singularity z = a. 
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If we introduce polar coordinates by letting z — a = ret and dz = ire®'d0, 


2m OF 20 
f da =| UE ajai] ab 2 ni (1.4.20) 
C 0 


z—a refi 0 


Note that the integrand becomes undefined at z = a. Furthermore, the answer is indepen- 
dent of the size of the circle. Our example suggests that when we have a closed contour 
integration, it is the behavior of the function within the contour rather than the exact shape 
of the closed contour that is of importance. We will return to this point in later sections. 


Problems 
1. Evaluate ¢,(z*)? dz around the circle |z| = 1 taken in the counterclockwise direction. 


2. Evaluate fo |z|? dz around the square with vertices at (0,0), (1,0), (1,1), and (0,1) taken 
in the counterclockwise direction. 


3. Evaluate fo |z|dz along the right half of the circle |z| = 1 from z = —i to z =i. 
4. Evaluate fo e* dz along the line y = x from (—1, —1) to (1,1). 
5. Evaluate [.,(z*)? dz along the line y = x? from (0,0) to (1,1). 


6. Evaluate fn z~'/? dz, where C is (a) the upper semicircle |z| = 1 and (b) the lower semi- 
circle |z| = 1. If z = re”, restrict —m < 0 < 7. Take both contours in the counterclockwise 


direction. 
1.5 THE CAUCHY-GOURSAT THEOREM 


In the previous section we showed how to evaluate line integrations by brute-force 
reduction to real-valued integrals. In general, this direct approach is quite difficult and we 
would like to apply some of the deeper properties of complex analysis to work smarter. In 
the remaining portions of this chapter, we introduce several theorems that will do just that. 

If we scan over the examples worked in the previous section, we see considerable differ- 
ences when the function was analytic inside and on the contour and when it was not. We 
may formalize this anecdotal evidence into the following theorem: 


Cauchy-Goursat theorem:? Let f(z) be analytic in a domain D and let C be a simple 
Jordan curve? inside D so that f(z) is analytic on and inside of C. Then fo f(z) dz = 0. 


Proof: Let C denote the contour around which we will integrate w = f(z). We divide the 
region within C into a series of infinitesimal rectangles. See Figure 1.5.1. The integration 


2 Goursat, E., 1900: Sur la définition générale des fonctions analytiques, d’après Cauchy. Trans. Am. 
Math. Soc., 1, 14-16. 


3 A Jordan curve is a simply closed curve. It looks like a closed loop that does not cross itself. See 
Figure 1.5.2. 
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Figure 1.5.1: Diagram used in proving the Cauchy-Goursat theorem. 


around each rectangle equals the product of the average value of w on each side and its 
length, 


Ow dx dw, , dw d(iy) 
Gad d+ [ws ae” + By) 2 Jaw 

l _Owdr ðw l _ 3w d(iy) 

AET t gy | at lot a e ai 
Ow Ow)... 

Substituting w = u+ iv into Equation 1.5.1, 
Ow Ow ðu Ov ðv Ou 
= Hi ‘ 1.5.2 

Ox iðy (= =) (Z5) (a 


Because the function is analytic, the right side of Equation 1.5.1 and Equation 1.5.2 equals 
zero. Thus, the integration around each of these rectangles also equals zero. 

We note next that in integrating around adjoining rectangles, we transverse each side 
in opposite directions, the net result being equivalent to integrating around the outer curve 
C. We therefore arrive at the result fo f(z) dz = 0, where f(z) is analytic within and on 
the closed contour. 


The Cauchy-Goursat theorem has several useful implications. Suppose that we have a 
domain where f(z) is analytic. Within this domain, let us evaluate a line integral from point 
A to B along two different contours Cı and C2. Then, the integral around the closed contour 
formed by integrating along C, and then back along C2, only in the opposite direction, is 


f f(z)dz = T f(z)dz — f(z)dz=0 (1.5.3) 
C Ci C2 


or 


f(z)dz= f(z) dz. (1.5.4) 
Cı C2 
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4 (a) (b) 


Figure 1.5.2: Examples of a (a) simply closed curve and (b) not simply closed curve. 


Because C; and Co are completely arbitrary, we have the result that if, in a domain, f(z) 
is analytic, the integral between any two points within the domain is path independent. 

One obvious advantage of path independence is the ability to choose the contour so that 
the computations are made easier. This obvious choice immediately leads to the following 
principle: 


The principle of deformation of contours: The value of a line integral of an analytic 
function around any simple closed contour remains unchanged if we deform the contour in 
such a manner that we do not pass over a nonanalytic point. 


e Example 1.5.1 
Let us integrate f(z) = z7! around the closed contour C in the counterclockwise 
direction. This contour consists of a square, centered on the origin, with vertices at (1,1), 
(1,-1), (—1,1), and (—1,—1). 

The direct integration of ¢,z~'dz around the original contour is very cumbersome. 
However, because the integrand is analytic everywhere except at the origin, we may deform 
the origin contour into a circle of radius r, centered on the origin. Then, z = re® and 


dz = rie"d@ so that j z 5 
d T $ . T 
¢ a =| “do = if dO = 2ni. (1.5.5) 
c? o re 0 


The point here is that no matter how bizarre the contour is, as long as it encircles the origin 
and is a simply closed contour, we can deform it into a circle and we get the same answer 
for the contour integral. This suggests that it is not the shape of the closed contour that 
makes the difference but whether we enclose any singularities (points where f(z) becomes 
undefined) that matters. We shall return to this idea many times in the next few sections. 


Finally, suppose that we have a function f(z) such that f(z) is analytic in some domain. 
Furthermore, let us introduce the analytic function F(z) such that f(z) = F’(z). We would 
like to evaluate f? f(z) dz in terms of F(z). 

We begin by noting that we can represent F, f as F(z) = U +iV and f(z) = u + iv. 
From Example 1.3.28 we have that u = U, and v = Vz. Therefore, 


b b b b 
J f@a= f wrivar+iay = f Us de — Vady +i | V, dx + U, dy (1.5.6) 


a a 
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= f Usde+ Uydu ti f Vode + Vyay= farsi f av = F0) - FO) 
(1.5.7) 
or b 
J f(z) dz = F(b) — F(a). (1.5.8) 


Equation 1.5.8 is the complex variable form of the fundamental theorem of calculus. Thus, 
if we can find the antiderivative of a function f(z) that is analytic within a specific region, 
we can evaluate the integral by evaluating the antiderivative at the endpoints for any curves 
within that region. 


e Example 1.5.2 
Let us evaluate Ai z sin(z?) dz. 


The integrand f(z) = z sin(z?) is an entire function and its antiderivative equals 
—4cos(z?). Therefore, 


ie z sin(z?)dz = —4 cos(z?)|7" = $[cos(0) — cos(—x?)] = $[1 — cos(7?)]. (1.5.9) 


Problems 


For the following integrals, show that they are path independent and determine the value 
of the integral: 


24377 20 T 2i 
1. / e`” dz 2. | [e* — cos(z)] dz 3. | sin?(z) dz 4. / (z+ 1)dz 
1 0 0 


ri —i 


2+2i 2i i 1+i 
5. J (2? —z+8)dz 6. fia —i)z?+2iz—4]dz 7. J 2 cos(z2?)dz 8. / ze” dz 
1 1 0 i 
1.6 CAUCHY’S INTEGRAL FORMULA 


In the previous section, our examples suggested that the presence of a singularity 
within a contour really determines the value of a closed contour integral. Continuing with 
this idea, let us consider a class of closed contour integrals that explicitly contains a single 
singularity within the contour, namely fo g(z) dz, where g(z) = f(z)/(z— zo), and f(z) is 
analytic within and on the contour C. We closed the contour in the positive sense where 
the enclosed area lies to your left as you move along the contour. 

We begin by examining a closed contour integral where the closed contour consists of 
the C1, C2, C3, and C4 as shown in Figure 1.6.1. The gap or cut between C2 and Cy is very 
small. Because g(z) is analytic within and on the closed integral, we have that 


L(2) dz + (2) dz + A dz + EN dz = Q. (1.6.1) 
C, 27T Z0 Co ~ — Z0 C3 ~~ Z0 Ca Z T 70 


It can be shown that the contribution to the integral from the path C2 going into the 
singularity cancels the contribution from the path C4 going away from the singularity as 
the gap between them vanishes. Because f(z) is analytic at zo, we can approximate its 
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y Cy 


Figure 1.6.1: Diagram used to prove Cauchy’s integral formula. 


value on C3 by f(z) = f(zo) + 46(z), where 6 is a small quantity. Substituting into Equation 


1.6.1, 
L a= -ta f : dz f da dz. (1.6.2) 


Cı Z 7 Z0 3 ~ — Z0 3 ~ — Z0 


Consequently, as the gap between C2 and C4 vanishes, the contour C1 becomes the closed 
contour C so that Equation 1.6.2 may be written 
(2) ; I 
—— dz = 2rif (zo) + i ô dé, (1.6.3) 
C Z — Zo 0 

where we set z — zo = ee” and dz = iee*'dé. 

Let M denote the value of the integral on the right side of Equation 1.6.3 and A equal 
the greatest value of the modulus of 6 along the circle. Then 


2m 


2m 
|M| < i ldo < | Add =2nA. (1.6.4) 
0 0 


As the radius of the circle diminishes to zero, A also diminishes to zero. Therefore, |M|, 
which is positive, becomes less than any finite quantity, however small, and M itself equals 
zero. Thus, we have that 


soa eee) ay, (1.6.5) 


271 Jo Zz — Zo 


This equation is Cauchy’s integral formula. By taking n derivatives of Equation 1.6.5, we 
can extend Cauchy’s integral formula* to 


f™ (z) = TI Ie) de (1.6.6) 


2ri 


4 See Carrier, G. F., M. Krook, and C. E. Pearson, 1966: Functions of a Complex Variable: Theory 
and Technique. McGraw-Hill, pp. 39—40 for the proof. 
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for n = 1,2,3,.... For computing integrals, it is convenient to rewrite Equation 1.6.6 as 
f(z) Qri 
dz = (") (29). 1.6.7 
$c © = FM) (1.6.7) 


e Example 1.6.1 


Let us find the value of the integral 


gO) aa 
£ Gmie (1.6.8) 


where C is the circle |z| = 5. Using partial fractions, 


= 1.6.9 
(z-l1)(2-2) 2-2 2-1 ( ) 
and 
f 2 AOE) a f ra f y (1.6.10) 
c (z — 1)(z2 — 2) c z-2 c 2-1 
By Cauchy’s integral formula with zp = 2 and z = 1, 
cos(1z) 
dz = 2ni cos(27) = 27%, (1.6.11) 
C 27 2 
and 
$ SA E E (1.6.12) 
C 27 1 


because zp = 1 and zp = 2 lie inside C and cos(7z) is analytic there. Thus the required 
integral has the value 


cos(1z) 248 
aes = Ani. (1.6.13) 


e Example 1.6.2 


Let us use Cauchy’s integral formula to evaluate 
= f on a A (1.6.14) 


We need to convert Equation 1.6.14 into the form Equation 1.6.7. To do this, we rewrite 
Equation 1.6.14 as 


a_i VEE) 4 
f (z — 1)?(z — 3) a f (z—1)? dz. (1.6.15) 
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Therefore, f(z) = e*/(z — 3), n = 1, and z = 1. The function f(z) is analytic within 
the closed contour because the point z = 3 lies outside of the contour. Applying Cauchy’s 
integral formula, 


f e? d _ 2mi d e* 
ia (z — 1)?(z = 3) = I dz\z—3 


3rie 


2 
(1.6.16) 


Project: Computing Derivatives of Any Order of a Complex or Real Function 


The most common technique for computing a derivative is finite differencing. Recently 
Mahajerin and Burgess® showed how Cauchy’s integral formula can be used to compute the 
derivatives of any order of a complex or real function via numerical quadrature. In this 
project you will derive the algorithm, write code implementing it, and finally test it. 


Step 1: Consider the complex function f(z) = u + iv, which is analytic inside the closed 
circular contour C of radius R centered at zo. Using Cauchy’s integral formula, show that 


n! 


2nR” 


f(z) = I ee: y) + iv(x, y)|[cos(nd) — isin(nd)] d9, 


where x = xo + Rcos(0), and y = yo + Rsin(0). 


Step 2: Using five-point Gaussian quadrature, write code to implement the results from 
Step 1. 


Step 3: Test out this scheme by finding the first, sixth, and eleventh derivative of f(x) = 
8x/(x? + 4) for x = 2. The exact answers are 0, 2.8125, and 1218.164, respectively. What 
is the maximum value of R? How does the accuracy vary with the number of subdivisions 
used in the numerical integration? Is the algorithm sensitive to the value of R and the 
number of subdivisions? For a fixed number of subdivisions, is there an optimal R? 


Problems 


Use Cauchy’s integral formula to evaluate the following integrals. Assume all of the contours 
are in the positive sense. 


sin? (z) sin (z) 1 
1. f ane dz 2. f Gade SE dz 3. f E EEN 4) dz 


4. f tants) dz 5. E = ye 6p sr 
i. 


z? +1 z? z 
7. f ~ dz 8. f — dz 9. — dz 
zija l zj=2 (z =.1) =2 (z + i)’ 
"i 2 Oh 
10. f a LL. f Eteo ay 12. dz 
ia 2t j=1 2(2? — 3) ies (2 — E T 


5 Mahajerin, E., and G. Burgess, 1993: An algorithm for computing derivatives of any order of a complex 
or real function. Computers & Struct., 49, 385-387. 
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1.7 TAYLOR AND LAURENT EXPANSIONS AND SINGULARITIES 


In the previous section we showed what a crucial role singularities play in complex 
integration. Before we can find the most general way of computing a closed complex integral, 
our understanding of singularities must deepen. For this, we employ power series. 

One reason why power series are so important is their ability to provide locally a general 
representation of a function even when its arguments are complex. For example, when we 
were introduced to trigonometric functions in high school, it was in the context of a right 
triangle and a real angle. However, when the argument becomes complex, this geometrical 
description disappears and power series provide a formalism for defining the trigonometric 
functions, regardless of the nature of the argument. 

Let us begin our analysis by considering the complex function f(z), which is analytic 
everywhere on the boundary, and the interior of a circle whose center is at z = zp. Then, if 
z denotes any point within the circle, we have from Cauchy’s integral formula that 


_ 1 TAG) ches ek IO 1 
ORE Pe E Zo f (z — 20)/(¢ 5a, E 


where C denotes the closed contour. Expanding the bracketed term as a geometric series, 
we find that 


f(z)= : p FO d+ (z a) f A dott ema)" f AS ac. 


Ori C — zo — 2%)? 
(1.7.2) 
Applying Cauchy’s integral formula to each integral in Equation 1.7.2, we finally obtain 
_ (z — 20) «y (z — z0)” (n) 
fasika fake a (1.7.3) 


or the familiar formula for a Taylor expansion. Consequently, we can expand any analytic 
function into a Taylor series. Interestingly, the radius of convergence! of this series may be 
shown to be the distance between zp and the nearest nonanalytic point of f(z). 
e Example 1.7.1 

Let us find the expansion of f(z) = sin(z) about the point zp = 0. 

Because f(z) is an entire function, we can construct a Taylor expansion anywhere on 
the complex plane. For zọ = 0, 


f(D = F0) + Oz + Hf" OL + Hf" (Oe2e+---. (1.7.4) 


Because f(0) =0, f’(0) =1, f”(0) =0, f’”(0) = —1 and so forth, 


O a a (1.7.5) 


Because sin(z) is an entire function, the radius of convergence is |z — 0| < 00, i.e., all z. 


6 A positive number h such that the series diverges for |z — zo| > h but converges absolutely for 
|z — zo| < h. 
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Figure 1.7.1: Contour used in deriving the Laurent expansion. 
e Example 1.7.2 


Let us find the expansion of f(z) = 1/(1 — z) about the point zo = 0. 
From the formula for a Taylor expansion, 


F = FO) + SO) + Hf" O02 + Hf" (V2 +--. (1.7.6) 


Because f™ (0) = n!, we find that 


fie) =1l4z24 274 8444-5 i (1.7.7) 


Equation 1.7.7 is the familiar result for a geometric series. Because the only nonanalytic 
point is at z = 1, the radius of convergence is |z — 0| < 1, the unit circle centered at z = 0. 


Consider now the situation where we draw two concentric circles about some arbitrary 
point zọ; we denote the outer circle by C while we denote the inner circle by C1. See 
Figure 1.7.1. Let us assume that f(z) is analytic inside the annulus between the two circles. 
Outside of this area, the function may or may not be analytic. Within the annulus we pick 
a point z and construct a small circle around it, denoting the circle by C2. As the gap or 
cut in the annulus becomes infinitesimally small, the line integrals that connect the circle 
Co to Cı and C sum to zero, leaving 


¢ IO) acu FO d¢ 4 FO) dC. (1.7.8) 
c¢=z ci 5z? Ca C77 
Because f(¢) is analytic everywhere within C2, 
aa g 10. 
Qrif (z) =$ Fag (1.7.9) 


Using the relationship: 


f OPA iC) as Fenty 
TE Gra 
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Equation 1.7.8 becomes 


nee aif. O w+ if Ma. (1.7.11) 


Now, 


= = (1.7.12) 


Ase Z— zo Z— zo RER rN 
=z h+ (422) + (42) ! (=) t | (1.7.13) 


where |z — zo|/|¢ — zo| < 1 and 


EC z=2%2-ÇFæ z-z 1=(Ç= z)/(2- z0) (1.7.14) 
2 n 
= z i t (=) H (=) eo (=) +e, (1.7.15) 
Z — 20 Z — zo Z— 2 z-z 


where |¢ — zo|/|z — zo| < 1. Upon substituting these expressions into Equation 1.7.11, 


re) =| Lf AO ge aay ule des 


2ri Ja ¢ — zo 201 — zo) 
eaa O 
a e | 
1 1 1 
+| Sem f Okt apm f, MOC aa 
TE L [Hote soe + (1.7.16) 
(z — zo)” Qri 
or 
a a2 | an eee z— RE pA POE 
Oe ea G-a a a ae Pak es 
(1.7.17) 
Equation 1.7.17 is a Laurent expansion.” If f(z) is analytic at zo, then ay = a2 = +--+ = an 


- = 0 and the Laurent expansion reduces to a Taylor expansion. If zo is a singularity of 
f(z), then the Laurent expansion includes both positive and negative powers. The coefficient 
of the (z — 29)~+ term, aj, is the residue, for reasons that will appear in the next section. 

Unlike the Taylor series, a Laurent series provides no straightforward method for ob- 
taining the coefficients. For the remaining portions of this section we illustrate their con- 
struction. These techniques include replacing a function by its appropriate power series, 
the use of geometric series to expand the denominator, and the use of algebraic tricks to 
assist in applying the first two methods. 


T Laurent, M., 1843: Extension du théorème de M. Cauchy relatif à la convergence du développement 
d’une fonction suivant les puissances ascendantes de la variable x. C. R. l’Acad. Sci., 17, 938-942. 
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e Example 1.7.3 


Laurent expansions provide a formalism for the classification of singularities of a func- 
tion. Isolated singularities fall into three types; they are as follows: 


e Essential Singularity: Consider the function f(z) = cos(1/z). Using the expansion for 


cosine, 
1 1 1 1 
cos ( ) =1 + P (1.7.18) 
zZ 


2lz2 4lz4  6lz6 


for 0 < |z| < oo. Note that this series never truncates in the inverse powers of z. Essential 
singularities have Laurent expansions, which have an infinite number of inverse powers of 
z — zo. The value of the residue for this essential singularity at z = 0 is zero. 


e Removable Singularity: Consider the function f(z) = sin(z)/z. This function has a 
singularity at z = 0. Upon applying the expansion for sine, 


j 1 gh tye’ G8 2 A 6 48 
sins) =(2 Reg ag La oi) = o pE a eo a ETTO) 


for all z, if the division is permissible. We made f(z) analytic by defining it by Equation 
1.7.19 and, in the process, removed the singularity. The residue for a removable singularity 
always equals zero. 


e Pole of order n: Consider the function 


1 


ee 1.7.20 
This function has two singularities: one at z = 1 and the other at z = —1. We shall only 
consider the case z = 1. After a little algebra, 
1 1 1 1 1 
= = 1.7.21 
PC) (z—-1)8 2+(z—1) 2(¢-1)? 14+(¢-1)/2 ey) 
1 1 z-1 (z-1)? (z-1)} 
= 1 pees 1.7.22 
2(z-1)8 2 4 8 ta?) 
1 1 1 J 
= H free (1.7.23) 


2(z— 1) 4(z-1)? 8&(z-1) 16 

for 0 < |z—1| < 2. Because the largest inverse (negative) power is three, the singularity 
at z = 1 is a third-order pole; the value of the residue is 1/8. Generally, we refer to a 
first-order pole as a simple pole. 


e Example 1.7.4 
Let us find the Laurent expansion for 


(1.7.24) 


about the point z = 1. 
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We begin by rewriting f(z) as 


1+(z-1) 1 1+(z-1) 


fj= (z — 1)[-2 + (z — 1)] = 2 (z— 1)[1 — (z — 1)] (1.7.25) 
= apa (z = 1) 1 pel De EN. 
Gay te ee (1.7.26) 
a a a 3 , 


provided 0 < |z — 1| < 2. Therefore we have a simple pole at z = 1 and the value of the 
residue is —1/2. A similar procedure would yield the Laurent expansion about z = 3. 


e Example 1.7.5 


Let us find the Laurent expansion for 


zr +27” 
= 0 >0 1.7.28 
Fl) z2 — 2z cosh(a) + 1’ re es ( ) 
about the point z = 0. 
We begin by rewriting f(z) as 
n ae 1 rA E A +z” 
= = . 1.7.29 
Fl) (z—e%)(z— e7) 2sinh(a) ( z— e% z — e7% ) ( ) 
Because 
1 e 7 
= ae (1+ zet +2 e7? p. 1.7.30 
e e e-* (1+ze 2e?) (1.7.30) 
if |z| < e% and 
1 Q 
= f = —e* (1 + ze% + 276? + ---) (1.7.31) 
Z— Ee 7% 1 — ze® 


if |z| < e79, 
a 


ETTO 


=Q 


2sinh(a) (2 


if |z| < e77. Clearly we have an nth-order pole at z = 0. The residue, the coefficient of all 
of the z~! terms in Equation 1.7.32, is found directly and equals 


_ sinh(na) 


Res| f(z); 0] = aha (1.7.33) 


For complicated complex functions, it is very difficult to determine the nature of the 
singularities by finding the complete Laurent expansion, and we must try another method. 
We shall call it “a poor man’s Laurent expansion.” The idea behind this method is the 
fact that we generally need only the first few terms of the Laurent expansion to discover 
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its nature. Consequently, we compute these terms through the application of power series 
where we retain only the leading terms. Consider the following example. 


e Example 1.7.6 


Let us discover the nature of the singularity at z = 0 of the function 


tz 


f(z) = (1.7.34) 


zsinh(az)’ 


where a and ¢ are real. 
We begin by replacing the exponential and hyperbolic sine by their Taylor expansion 
about z = 0. Then a 
Ll+tz+t?27/2+.--- 
f(zZ) = 353 
z(az + a3z3/6+---) 


(1.7.35) 
Factoring out az in the denominator, 


l+tz+t?27/2+--- 
f(2) = az?(1+a?z?/6+---)’ ea 


Within the parentheses, all of the terms except the leading one are small. Therefore, by 
long division, we formally have that 


1 
f(z)= =a +tz+t?27/2+---)(1—a?2?/6+---) (1.7.37) 
1 1 t t? — a? 
EEN EE Pe are E eee Ee. eae) 
az? az? az 6a 


Thus, we have a second-order pole at z = 0 and the residue equals t/a. 
Problems 
1. Find the Taylor expansion of f(z) = (1 — z)~? about the point z = 0. 


2. Find the Taylor expansion of f(z) = (z — 1)e* about the point z = 1. (Hint: Don’t find 
the expansion by taking derivatives.) 


By constructing a Laurent expansion, describe the type of singularity and give the residue 
at zo for each of the following functions: 


3. f(z) = ze“ V*, xy =0 4. f(z) =z 3sin?(z); 2 =0 
5. f(z) = cosh) zy =0 6. f(z) = T z= -2 
T f= S44; a =0 8 fle)= ao asti 
9. fle) = y x =2 10. f( yee, 2 =0 
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Figure 1.8.1: Contour used in deriving the residue theorem. 
1.8 THEORY OF RESIDUES 


Having shown that around any singularity we may construct a Laurent expansion, 
we now use this result in the integration of closed complex integrals. Consider a closed 
contour in which the function f(z) has a number of isolated singularities. As we did in the 
case of Cauchy’s integral formula, we introduce a new contour C” that excludes all of the 
singularities because they are isolated. See Figure 1.8.1. Therefore, 


f f(z)dz — fla)dz—-— $ f(z)dz = f(z)dz =0. (1.8.1) 
C Cı Cn C’ 


Consider now the mth integral, where 1 < m < n. Constructing a Laurent expansion for 
the function f(z) at the isolated singularity z = Zm, this integral equals 


£ f(z)dz= Daf ear = dz+ 


Because (z — zm)“ is an entire function if k > 0, the integrals equal zero for each term in 
the second summation. We use Cauchy’s integral formula to evaluate the remaining terms. 
The analytic function in the numerator is 1. Because d*~!(1)/dz*~! = 0 if k > 1, all of 
the terms vanish except for k = 1. In that case, the integral equals 271a1, where a, is the 
value of the residue for that particular singularity. Applying this approach to each of the 
singularities, we obtain the following: 


Co 


Z — Zm)" Z. .8. 
af í ha (1.8.2) 


m 


k= 


Cauchy’s residue theorem:® If f(z) is analytic inside and on a closed contour C (taken 
in the positive sense) except at points 21, 22,..., 2n where f(z) has singularities, then 


TOL = ani > Res[ f(z); z;], (1.8.3) 


8 See Mitrinovié, D. S., and J. D. Keékié, 1984: The Cauchy Method of Residues: Theory and Ap- 
plications. D. Reidel Publishing, 361 pp. Section 10.3 gives the historical development of the residue 
theorem. 
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where Res[f(z);z;] denotes the residue of the jth isolated singularity of f(z) located at 


e Example 1.8.1 


Let us compute Fina z?/(z +1) dz by the residue theorem, assuming that we take the 
contour in the positive sense. 

Because the contour is a circle of radius 2, centered on the origin, the singularity at 
z = —1 lies within the contour. If the singularity were not inside the contour, then the 
integrand would have been analytic inside and on the contour C. In this case, the answer 
would then be zero by the Cauchy-Goursat theorem. 

Returning to the original problem, we construct the Laurent expansion for the integrand 
around the point z = 1 by noting that 


z? _ [z@+)-1? _ 1 


= = 24 + 1). 1.8.4 
z+1 z+1 z+1 (2 ) ( ) 
The singularity at z = —1 is asimple pole and by inspection, the value of the residue equals 
1. Therefore, 
x 
f dz = 2ri. (1.8.5) 
Iz=2 7+1 


As it presently stands, it would appear that we must always construct a Laurent expan- 
sion for each singularity if we wish to use the residue theorem. This becomes increasingly 
difficult as the structure of the integrand becomes more complicated. In the following 
paragraphs we show several techniques that avoid this problem in practice. 

We begin by noting that many functions which we will encounter consist of the ratio of 
two polynomials, i.e., rational functions: f(z) = g(z)/h(z). Generally, we can write h(z) as 
(z— 2)" (z— 29)? ---. Here we assumed that we divided out any common factors between 
g(z) and h(z) so that g(z) does not vanish at 21, z2,.... Clearly 2, z2,..., are singularities 
of f(z). Further analysis shows that the nature of the singularities are a pole of order mı 
at z = z1, a pole of order mz at z = z2, and so forth. 

Having found the nature and location of the singularity, we compute the residue as 
follows. Suppose that we have a pole of order n. Then we know that its Laurent expansion 
8 an an-1 


NN (z — zo)” | (z — 2)"-1 Fe bo + bi(z— 20) +++. (1.8.6) 


Multiplying both sides of Equation 1.8.6 by (z — 20)”, 


F(z) = (z — 20)" f(z) = an +an-1 (2 — 20) +++ +b0(2 — 20)" + bi (z — 20)" T? +++. (1.8.7) 
Because F(z) is analytic at z = zo, it has the Taylor expansion 


(n-1)(z 
F(z) = F(z) + F' (zo) (z — zo) +4 ag pT a ee (1.8.8) 


Matching powers of z — zo in Equation 1.8.7 and Equation 1.8.8, the residue equals 


FO-) (z0) 


Res| f(z); zo] = a1 = “Gan” 


(1.8.9) 
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Substituting in F(z) = (z — zo)" f(z), we can compute the residue of a pole of order n by 


det 


Res[f(z); zj] = (n=1)! as dzn-1 


je- arro]. (1.8.10) 


For a simple pole, Equation 1.8.10 simplifies to 


Res[f (z); z;] = lim (z — z;)f (z). (1.8.11) 


Z> Zj 


Quite often, f(z) = p(z)/q(z). From lHôpital’s rule, it follows that Equation 1.8.11 be- 
comes 


Res[f(z); z] = AN (1.8.12) 


Recall that these formulas work only for finite-order poles. For an essential singularity we 
must compute the residue from its Laurent expansion; however, essential singularities are 
very rare in applications. 


e Example 1.8.2 


Let us evaluate 


e 
> dz, 1.8.13 
$ Pte ( ) 
where C is any contour that includes both poles at z = +ai and is in the positive sense. 
From Cauchy’s residue theorem, 
et” et” et? 
cz“ +a zt +a zt +a 
The singularities at z = tai are simple poles. The corresponding residues are 
e” e” ee 
Res| ———;ai | = li ) = 1.8.15 
(3 + a2 ci) ae a (z—ai)(z+at) 2ia ( ) 
and : 
e7 e7 e? 
Res =a S li Hai : ~ = —. 1.8.16 
(3 +a?’ ci) ee ay (z — ai)(z + at) 2ia ( ) 
Consequently, 


e7 2 pa a Qa. 
f, Pig dz= re (et -—e7*) = eG sinh(a). (1.8.17) 
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e Example 1.8.3 


Let us evaluate 


1 et 
d 1.8.18 
Qi $ z2(z22 + 2z +2) í ( ) 


where C includes all of the singularities and is in the positive sense. 


The integrand has a second-order pole at z = 0 and two simple poles at z = —1 +1, 
which are the roots of z? + 2z + 2 = 0. Therefore, the residue at z = 0 is 
el 1 d et 
R of =i 0)? 1.8.19 
| sera | 240 Tele ) al) ( ) 
te”? (2z + 2)e” t—1 
tin | aS (22 + 22 +2)? 2 ( ) 
The residue at z = —1 + i is 
et et” 
R ;—l1 +| = li 1+i 1.8.21 
“| sorry i iim, le ( Ne + 2z +2) ( ) 
, et , z+1-i 
E (i, z2 (i, A) (1:942) 
exp|(—1 +i)t] exp|(—1 + i)t] 
= = : 1.8.23 
2i(—1 +i)? 4 ( ) 
Similarly, the residue at z = —1—7 is 
et et 
;—l—iļ| = li 1—:ż 1.8.24 
oe 22(z2 +2242)’ i| ae ( ò) z2(z22 + 2z +2) ( ) 
ee z+1+i 
= (im 3) (im, 2422+ 5) (2:920) 
exp|(—1 — i)t] exp|(—1 — i)t] 
= = : 1.8.26 
(—2%)(—1 — i)? 4 ( ) 


Then by the residue theorem, 


1 f et J R et 6 R et j ; 
= Res ; + Res ; H2 
2ri Jo 2(2? + 2z + 2) á 8 z?(z22 +2242)’ 2?(z2 +22 4 2) 
el? 

;—l— i 1.8.27 
+ Res| a i ( ) 
t—1 _ exp|(—1+i)t] | exp[(—1-— i)t] 

a a 4 


= 4 [t — 1 + e™ cos(t)] . (1.8.29) 


4 T 
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Problems 


Assuming that all of the following closed contours are in the positive sense, use the residue 
theorem to evaluate the following integrals: 


z+1 (z+ 4) p} 1 
1. —— d 2. —— ~ d 3. — d 
f z 2z * L 244523 o ian 1 — € ‘ 
2 —4 3 
a g Z“ dz 5. g dz o. f zel dz, n>0 
lzļ=2 (2 — 1) Izļ=2 24-1 [gia 


2 


2 + 4cos(rz) z+1 dz 
7g e!/? cos(1/z) dz 8. g ———— dz 9 g - 
jz|=1 (1/2) giana 2 1)? [z-1J=2 2 — 1 sin(rz) 


Hint for Problem 9: sin(7z) = — sin[r(z — 1)] and z +1 = (z — 1) + 2. 
1.9 EVALUATION OF REAL DEFINITE INTEGRALS 

One of the important applications of the theory of residues consists of the evaluation of 
certain types of real definite integrals. Similar techniques apply when the integrand contains 
a sine or cosine. 


e Example 1.9.1 


Let us evaluate the integral 


f OR d — 1 J © da (1.9.1) 
o z?+1 2j œ 27417 E 
This integration occurs along the real axis. In terms of complex variables, we can rewrite 
Equation 1.9.1 as 
S 1 
J he = / aes (1.9.2) 
o z?+1 Belg Seo 


where the contour C; is the line S(z) = 0. However, the use of the residue theorem requires 
an integration along a closed contour. Let us choose the one pictured in Figure 1.9.1. Then 


f dz z dz -| dz (1.9.3) 
cz +1 o, Z? +1 o Z2? +L R 


where C denotes the complete closed contour and C2 denotes the integration path along 
a semicircle at infinity. Clearly we want the second integral on the right side of Equation 
1.9.3 to vanish; otherwise, our choice of the contour C2 is poor. Because z = Re® and 


dz = iRe™ dé, 
dz T  iRexp(62) A R 
=|= —— < do| < dé 1.9.4 
[. (as lk | l 1 + R? exp(20i) Tjo R-1 ’ ( ) 


which tends to zero as R + oo. On the other hand, the residue theorem gives 


dz 
cz? +1 


1 i 1 
= 2riRes( i) = 2ri lim 5 I = nix 5 =n. (1.9.5) 
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Figure 1.9.1: Contour used in evaluating the integral, Equation 1.9.1. 


J. E (1.9.6) 


etl 2 
Note that we only evaluated the residue in the upper half-plane because it is the only one 
inside the contour. 


Therefore, 


This example illustrates the basic concepts of evaluating definite integrals by the residue 
theorem. We introduce a closed contour that includes the real axis and an additional 
contour. We must then evaluate the integral along this additional contour as well as the 
closed contour integral. If we properly choose our closed contour, this additional integral 
vanishes. For certain classes of general integrals, we shall now show that this additional 
contour is a circular arc at infinity. 


Theorem: If, on a circular arc Cr with a radius R and center at the origin, zf(z) > 0 
uniformly with |z| € Cr and as R > œ, then 
lim f(z) dz =0. (1.9.7) 
R- 00 Cr 


The proof is as follows: If |zf(z)| < Mr, then |f(z)| < Mpr/R. Because the length of 
Cr is aR, where a is the subtended angle, 


(z) dz 


M 
< —aR=aMp > 0, (1.9.8) 
ey R 


because Mp —> 0 as R —> œ. 


e Example 1.9.2 


A simple illustration of this theorem is the integral 


ae dx dz 
=-= s. (1.9.9) 
-œo T? al! Joz tz+1 


A quick check shows that z/(z? + z + 1) tends to zero uniformly as R > oo. Therefore, if 
we use the contour pictured in Figure 1.9.1, 


2 dx dz 1 
pa Sp E ,14 v3; 1.9.10 
| ae rere ve (so 20a ( ) 
1 20 
= 271 li —— } = —. 1.9.11 
ý Sa (= i) V3 ( ) 
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Figure 1.9.2: Contour used in evaluating the integral, Equation 1.9.13. 


e Example 1.9.3 


Let us evaluate 


[ A (1.9.12) 


xê +1 


In place of an infinite semicircle in the upper half-plane, consider the following integral 


dz 
$ ae (1.9.13) 


where we show the closed contour in Figure 1.9.2. We chose this contour for two reasons. 
First, we only have to evaluate one residue rather than the three enclosed in a traditional 
upper half-plane contour. Second, the contour integral along C3 simplifies to a particularly 
simple and useful form. 


Because the only enclosed singularity lies at z = e™*/®, 
dz 1 f z — emi/6 
= 95 Pes -e™/6)\) On; li Soe 
pas 2ri Re(- z FE ) = 2ri oY FI (1.9.14) 
1 Ti z; 
=2mi lim — =- e6, 1.9.15 
™ emie 68 et) 
Let us now evaluate Equation 1.9.12 along each of the legs of the contour: 
d S d 
f = = Tae (1.9.16) 
ar tA o +1 
d 
l -Z =0, (1.9.17) 
Gs z + 1 
because of Equation 1.9.7 and 
d 0 ni/3 q : © d 
I Sae =| ca -ers f er (1.9.18) 
c 2° +1 œ To +1 o «6 +1 


since z = re™/3, 
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Substituting into Equation 1.9.15, 


ree Tt oe 
1— o — _ "oTi/6 1.9.1 
( e o zê+1 3° te) 
or 
i me mi aieri/6 esate eT T (1.9.20) 
o 6+1 6 eTi/6 (eTi/6 — e=7i/6) 6sin(t/6) 3 


e Example 1.9.4 


Rectangular closed contours are best for the evaluation of integrals that involve hyper- 
bolic sines and cosines. To illustrate? this, let us evaluate the integral 


2 fo sin(az) sinh(x) dz = u sin(ax) sinh(z) dz = of sinh(a)e**” l 
o [b+ cosh(x)]? -œ [b + cosh(a)|? —oo [b + cosh(x)]? 
(1.9.21) 
where a > 0 and b> 1. 
We begin by determining the value of 


sinh(z)e** 
¢ Ha aa 
c [b + cosh(z)] 
about the closed contour shown in Figure 1.9.3. Writing this contour integral in terms of 
the four line segments that constitute the closed contour, we have 


sinh(z)e’” L sinh(z)e*? T sinh(z)e’?” 
£ [b + cosh(z)/2 ee f, [b + cosh(z)]? des) ca [b + cosh(2)] 4 


sinh(z)e** sinh(z)e** : 
H: [b + cosh(z)]? de [. [b + cosh(z)]? dz. (1.9.22) 


Because the integrand behaves as e~” as R — oo, the integrals along C2 and C4 vanish. 
On the other hand, 


sinh(z)e’4” -— fS -sinh a ye* 
i. [b + cosh(z)]? ee yes [b + cosh(x)]? on (1:928) 


and 


sinh(z)e’* -a i sinh(x)e’*” 
= ie d 1.9.24 
hrer LA ER 


9 This is a slight variation on a problem solved by Spyrou, K. J., B. Cotton, and B. Gurd, 2002: 
Analytical expressions of capsize boundary for a ship with roll bias in beam waves. J. Ship Res., 46, 
167-174. 
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(-R,2ni) C3 (R,2ni) 


(-R,0) Cı (R0) 


Figure 1.9.3: Rectangular closed contour used to obtain Equation 1.9.31. 


because cosh(x + 277) = cosh(x) and sinh(# + 2ri) = sinh(x). 

Within the closed contour C, we have a single singularity where b + cosh(zs) = 0 or 
e* = —b— Vb? — 1 or zs = ln(b + Vb? — 1 ) + ri. To discover the nature of this singularity, 
we expand b + cosh(z) in a Taylor expansion and find that 


b + cosh(z) = sinh(zs)(z — zs) + $ cosh(zs)(z — 2s)? +-+. (1.9.25) 


Therefore, we have a second-order pole at z = zs. Therefore, the value of the residue there 
is 


a E ooh | ~ ae, ae + a y =e) “eo? 
= no exp[ia cosh—1(6)). (1.9.27) 
Therefore, 
/ > sinh(@)e** 4, . —2raexp[—na + at cosh’ '(b)] _ maexplai cosh’ (b)] (1.9.28) 
-oo [b + cosh(z)]? (1 — e~?*) sinh(zs) Ve? —1 sinh(ra) ° 
because 


1 1 
sinh(z,) = b b2 —1+ z= b2- 1. 1.9.29 
mhle) = 5 |-b- VI+ ee (1.9.29) 


Substituting Equation 1.9.28 into Equation 1.9.21 yields 


a sin(ax) sinh(x) ira Tasinia cosh '(b)| (1.9.30) 
0 


[b + cosh(a)]? = 2b — 1 sinh(ra) 
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e Example 1.9.5 


The method of residues is also useful in the evaluation of definite integrals of the form 
ie F[sin(0), cos(@)|d6, where F is a quotient of polynomials in sin(@) and cos(@). For 
example, let us evaluate the integral!? 


27 3 
cos’ (0) 
I= a ; .9. 
I cg >l (1.9.31) 


We begin by introducing the complex variable z = et. This substitution yields the 


closed contour integral 
1 (z2? + 1)8 dz 
[= 1.9.32 
2i £ (22 + 1)? — 4a2z2 22’ ( ) 


where C is a circle of radius 1 taken in the positive sense. The integrand of Equation 1.9.32 
has five singularities: a second-order pole at z5 = 0 and simple poles located at 


z1 =—a— Va? -— 1l, z2 = —a + Va — 1, (1.9.33) 


z3 =a— ya?-—1l, and z4=a+ vya? -—1. (1.9.34) 


Only the singularities z2, z3, and z5 lie within C. Consequently, the value of J equals 277 
times the sum of the residues at these three singularities. The residues equal 


Res ay ;—a + vazi} 


22[(z2 + 1)? — 4a222]’ 


2 1 3 z ae | 
a te Se, Mie eee (1.9.35) 
z-a} -I Z? ee et Vata (27+ 1)? — 4072 
2 1 3 
= lm (et 4) (1.9.36) 
2—otVarai 429(2? + 1— 2a?) 
2 = I 1 3 
O a l (1.9.37) 
(2a? — 1 — 2av a? — 1 )(a? — 1 — ava? — 1) 
(2 ae 1)3 
R 4 2—1 
es{ 22[(22 + 1)? — 4a222]’ È 7 
2 1 3 os 2—1 
E N a, ee (1.9.38) 
z>a-va I 2? zoa- vami (2? +1) — 4072? 
2 1 3 
= lim cere (1.9.39) 
zza- Vai 423(z? + 1 — 2a?) 
264. 2_] 3 
= A a (1.9.40) 


(2a2 — 1 — 2a v'a? — 1 ) (a2 — 1 — ava? —1)’ 


10 Simplified version of an integral presented by Jiang, Q. F., and R. B. Smith, 2000: V-waves, bow 
shocks, and wakes in supercritical hydrostatic flow. J. Fluid Mech., 406, 27-53. 
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and 
2 3 
Res Cae, ; 
z2[(z22 + 1)? — 4a?z?] 
> d (27 + 1)8 
= lim 
240 dz | (z? +1)? — 4a?2z? 


6z[(z? + 1)* — 4a2z?(2? + 1)?] — 42(z? +1)3(z? +1 — 2a?) 
= lim 
z—0 (22 + 1)2 — 4a? z?]2 


Summing the residues, we obtain 0. Therefore, 


20 3 
cos’ (@) 
—— dl = 1. 
J cos? (0) — a? 0, fa 


Problems 


Use the residue theorem to verify the following integrals: 


Lf dx _ 7v2 2. | dx T 
0 


aé+1l. 4 oo (z2 +4r+5)2 2 
ia x dx T were ape T 
3. = 4. | ——dr=-— 
leo a 5 I a 6 
5 4 dx oT 6 if dx 5r 
“Jo (x2 +1)2 4 ` Jo (a2 +1)(a2 +4)? 288 
7. 
N z? dx B T AE 
oo (+a) +2) Wat P 
8. 
P t? T a—h 
dt=—|1 
I CENEO + @/h—1) al Va eee 
9. Show that 


7/2 dp T 
f -z = ; a> 0. 
o a+sin*(0) 2/a+a? 


Step 1: Convert the real integral into a closed contour integration: 


ae dé p Zz 
2p, ~ 2 2 2 z dz, 
0 a + sin (0) |z|=1 (z = 1) — daz 


where z = e®. 
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(1.9.41) 


(1.9.42) 


(1.9.43) 


(1.9.44) 


Step 2: Show that the integrand has four poles: z = +ya + v1 +a. Only two are located 


inside the contour: z1 = —ya + V1 +a and z2 = ya — v1 +a. 
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Step 3: Show that the corresponding residues are 


z z 1 


Res | ———.——;; 21 | = Res] — r; 22] = -— m. 
es| — 1)? — 4az?’ a esl zz — 1)? — 4az2’ 2 8vVa + a? 


Step 4: Obtain the final result by applying the residue theorem and the results from Step 
1 through Step 3. 


10. Show that 


| i 2 2 2 b > a> 0 
2 2 t , = s 
0 aî COS (8) + b? sin (0) 2a 


Step 1: Convert the real integral into a closed contour integration: 


m do d z ; 
= —4 A 
o a? cos? (0) + b2 sin? (0) jzjar (22 + 1)? — (22 — 1? 


where z = e®', 

Step 2: Show that the integrand has four simple poles located at z? = (b + a)/(b— a), and 
22 = (b—a)/(b+a). Only two are located inside the contour: Zz = (b — a)/(b + a), and 
zP =- b-a) /b +a). 


Step 3: Show that the corresponding residues are 


l á eee Res z „lL 
me ae ee ef SEEDED Bab’ 


Step 4: Obtain the final result by employing the residue theorem and the results from Step 
1 through Step 3. 


11. Show that 


™ sin? (0) T 
dO = 2—2 b : 
i a + bcos(6) b2 (a A ) f vrag 


Step 1: Convert the real integral into a closed contour integration: 


To n2 . 2_ 4)2 
f sin“ (0) do = f (z* — 1) ee 
o a+ bcos(A) 4 Jizjar b(2? + 1) + 2a2]2? 


where z = e®*. 


Step 2: Show that the integrand has a second-order pole at z = 0 and simple poles at 
z1,2 = (—a + Va? — b? ) /b. Only the poles located at z = 0 and z1 = (—a + Va? — b? ) /b 
lie within the closed contour. 


Step 3: Show that the corresponding residues are 


(z2 — 1)? 2a (22 — 1)? 2V a? — b? 
;0| = , and Res z| = ——. 
[b(z? + 1) + 2az]z? b2 [b(z2 + 1) + 2az]z? b2 


Res 
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Step 4: Obtain the final results by employing the residue theorem and Step 1 through Step 
3. 


12. Show that 


2m ind n 
e (=r) 
dO = 2r >= 1 = OFT 2c aiek 
i 1 + 2r cos(0) + r? TIZ r? ally e012, 


Step 1: Convert the real integral into a closed contour integration: 


2n ind n 
f e do = if : dz, 
o 14+ 2rcos(@) +r? jejar P(22 +1) + (1 +r?)z 
where z = e”. 


Step 2: Show that the integrand has two simple poles: z} = —r, and z- = —1/r. Why is 
the z4 pole the only one inside the contour? 


Step 3: Show that the corresponding residue is 


yn (—r)" 


r(2@+l+0+r)z2 *] lr 


Res | 


co i 


Step 4: Obtain the final result by using the residue theorem and Step 1 through Step 3. 


13. Show that 5 On) 
n 27 (2n)! 
+ 2n aE EN A 

i sin” (0) d (annl)2 


Step 1: Convert the real integral into a closed contour integration: 


2r =% 2 1)? 
modo- — f Gaal 
il sin?" (0) d9 = yam faa & 


where z = e", 
Step 2: Show that the integrand has a pole of order 2n + 1 at z = 0. 


Step 3: Because 


2n)!(—1)” 
(22 —1)?? = 24" mnl 4... 4 (2n)!(—1) Pires, 
nin! 
show that (2 j2 OnI) 
z—1)°" 2n)!(—1)” 
Res| z2n+1 0| = nin! i 


Step 4: Obtain the final result by using the residue theorem and Step 1 through Step 3. 
14. Show that 


[ cos(n@) ye ng Gar Va2—1)" 


 COs(9) +a a2 —1 
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Step 1: Convert the real integral into a closed contour integration: 


f cos(n0) do = i f 20 zT A 
-z cos(@) +a i Ji=1 27 +2az +1 


where z = e”. 

Step 2: Assume that n 4 0. Show that the integrand has an n-order pole at z = 0 and 
simple poles at 21,2 = —a + Va? — 1. Why is z271 = —a+ va? — 1 the only simple pole that 
lies inside the contour? 


Step 3: Because 


e KS 1 ge aig Vi E 
z2? +2az+1 2V2 —1 , 


show that 


gv gr gen 4 
Res( 42" 2] =í% . 
Z— Ry 


Step 4: Because 


= 2 3 
eet Pe 
Z-21 Zy ZY Zi Zi 
show that 
gage 1 fe ee ae 1 
Res( 10) = =e and Res(# if 0) = a = Oa 
Z— 2 zy Z— 22 25 


Step 5: Use the residue theorem plus Steps 1 through 4 to obtain the final result when 
n#0. 


Step 6: Redo the problem when n = 0. In this case we only have the pole at z = 21. 


15. Show that 


T cos(n@) ae 
eee eae ee na F 
J cosh(a) — cos(0) sinh(q) e", a#0, n>0 


Step 1: Convert the real integral into a closed contour integration: 


E cos(n0) do = 1 f erg” d 
o cosh(a)—cos(@) a Jj, 2? — 2zcosh(a) + 1 ee 


Oi 


where z = e 


Step 2: Show that the integrand has an n-order pole at z = 0 and simple poles at 21,2 = 
e“,e-*. Because a can be taken as positive without loss of generality, then only the poles 
located at z = 0 and z = e~® lie within the closed contour. 


Complex Variables 47 


Step 3: Because 


zr +z” = 1 2z gM 427” 
z2? — 2z cosh(a)+1 2sinh(a) \ z-e% z—er J’ 
show that the corresponding residues are 
Rali aapa ale Paa 
z? — 2z cosh(a) + 1 sinh(a) 
from Example 1.7.5 and 
Res z” +27” „e| =- Senay: 
z? — 2zcosh(a) + 1 sinh(a) 


Step 4: Use the residue theorem plus Steps 1 through 3 to finish the problem. 


16. Show that 


A r? T 

dx = ; 
o O-ro?” T Fal 
where a is real and not equal to zero. 


Step 1: Show that 


ie i dx = 5 f a d 
t = zZ 
0 (1 -_ x)? + a2 x2 2 c (1 = 27)? + a2 z2 , 


where C denotes a semicircle of infinite radius in the upper half of the complex plane. Along 
the real axis, the contour slightly above y = 0 when x < 0 and slightly below y = 0 when 
x>0. 


Step 2: Show that the poles of the integrand are simple and equal 
s(tV4—a? +|ali), if 0<|al <2, 


i (Jal aod). if “Dee lal: 


Zn = 


If |a| = 2, we have second-order poles at zn = +i. 
Step 3: Show that the residues for the poles in the upper half plane are 


aa To tei) a EVE + Halt 2 
Res| f(e): 3 4—a? + lal | = Save 


Ress): (lal + 


2 _ _i(la| + Va? —4)/2 
=] Javaa 
and 

Res[f(z);i] = —<. 


Step 4: Show that when you sum the residues for the cases for 0 < |a| < 2 and 2 < |a|, you 
obtain —i/(2|a]) 
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Step 5: Redo the calculation when |a| = 2. 


17. Evaluating the closed contour integral 


ela 
$ EUS RCD 
c cosh‘ (bz) 
around the rectangular contour with vertices at (20, 0), (—0o,0), (co, 7/b), and (—o0, m/b), 


show that pa Gi 
COS( axr Ta 
dx = b i 
[ cosh? (bx) 7T a sinh[ar /(2b)] eee 


Step 1: Show that 
ettz eltz ele 
—,—- dz = dz4 d 
$ cosh? (bz) L cosh? (bz) í [. cosh? (bz) 
ettz eitz 
+ —— dz + | —,——~ dz, 
J cosh? (bz) c, cosh? (bz) A 


where C1, C2, C3 and Cy are the contours along the bottom, right side, top, and left side 
of the rectangle. 


Step 2: Show that the integrals along C and C4 vanish. Why? 


Step 3: Show that ; 
[ nee e 
cı cosh“ (bz) —oo cosh“ (bx) 


i. 7 s dz = D. n dx. 
C, cosh“ (bz) -œ cosh“ (bz) 


Step 4: Setting zs = 7i/(2b), show that the Laurent expansion for e!“*/ cosh? (bz) at zs is 


and 


e7 e iae 


= + 
cosh? (bz) b2(z — zs)?  b?(z-— zs) 


1AZs5 1AZs5 


Hence, we have a second-order pole there. 


Step 5: Show that 


co iax 2 —ra/(2b) 
(i= e570) f Ss 
-œ cosh“ (bz) b? 


Step 6: Simplify Step 5 to obtain the desired result. 


18. Using the closed contour integral 


T 
zZ 
c cosh(z)cosh(z +a) ° 
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where C is a rectangular contour with vertices at (00,0), (—00,0), (co, 7), and (—00, 7), 
show!! that 


oe dx 2a/sinh(a), if a0, 
9 cosh(x) cosh(a + a) 


= 2 if a2. 


’ 


Step 1: If a #0, show that 
f f(z) dz = 2ri Res[f(z); z1] + 2ri Res[ f(z); z2], 
c 


where zı and z2 are simple poles with zı = mi/2 and z2 = —a + 7i/2, respectively. 
Step 2: Show that in this case, 


Ti 


Ti 1 
-anha and Res[f (z); z2] = (3 — a) A 


Res[f(z); 21] = 


Step 3: If a = 0, show that we a second-order pole located at z1 = mi/2 within the closed 
contour with 


; d [ 2(z— 7i/2)? _  d [n?(n t+ Ti/2) 
Res[ f(z); 21] Sere =| cosh” (z) | o TA A sinh’ (n) | 
ie d | n+ ri/2 
n>0 dn | (1 + 72/6 +---)? 
lim [e =-1. 
a +n?/6+:-)* 


II 


n-0 


Step 4: Denoting the contour along and parallel to the y-axis at x = oo as Cy and the 
contour along and parallel to the y-axis at x = —oo as C4, show that 


(z) dz > 0 as z —> 0, and (z)dz > 0 as z > —0O. 
C2 C4 


Step 5: Along the real axis, call it C1, show that 


sa x 
dz = d 
A (aia L cosh(x + a) Š 


while along the contour C3 (which runs parallel to the real axis but m units above it), 


ies J xr+ti 
C3 


oo cosh(x) cosh(a + a) 


Step 6: If a 40, show that 


i. dx 2rai 
—ri = 
oo cosh(x) cosh(x + a) sinh(a)’ 


11 See Yan, J. R., X. H. Yan, J. Q. You, and J. X. Zhong, 1993: On the interaction between two 
nonpropagating hydrodynamic solitons. Phys. Fluids A, 5, 1651-1656. 
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while for a = 0, show that 
af “ee 
-ri | a = TT. 
-o cosh* (x) 


t12 


19. During an electromagnetic calculation, Strutt’ needed to prove that 


G ) = 26 eee (n+ 5) (a-7)] 


<T. 
" cosh(on) nt 1)? RESH 


? 


Verify his proof by doing the following: 
Step 1: Using the residue theorem, show that 


1 sinh(xz) dz sinh(ox) a (ee sin [(n + $) z] 
2 fo 2 


cosh(1z) z — o "cosh(om) 


where Cy is a circular contour that includes the poles z = o and z, = +i (n+ 5) n= 
Ol Qc IN. 


Step 2: Show that in the limit of N — oo, the contour integral vanishes. Hint: Examine 
the behavior of z sinh(#z)/[(z — o) cosh(mz)] as |z| > oo. Use Equation 1.9.7 where Cp is 
the circular contour. 


Step 3: Break the infinite series in Step 1 into two parts and simplify. 


You would obtain the same series by computing the Fourier series of sinh(ax)/ cosh(o7) 
and using direct integration. 


1.10 CAUCHY’S PRINCIPAL VALUE INTEGRAL 


The conventional definition of the integral of a function f(x) of the real variable x over 
a finite interval a < x < b assumes that f(a) has a definite finite value at each point within 
the interval. We shall now extend this definition to cover cases when f(x) is infinite at a 
finite number of points within the interval. 

Consider the case when there is only one point c at which f(x) becomes infinite. If c 
is not an endpoint of the interval, we take two small positive numbers e€ and 7 and examine 


the expression 
c—eE b 
f f(x) ac+ | f(a) da. (1.10.1) 
a c+n 


If Equation 1.10.1 exists and tends to a unique limit as € and 7 tend to zero independently, 
we say that the improper integral of f(x) over the interval exists, its value being defined by 


b ee b 
i. fla) de = tim | f(x) ) dz + lim, f(x)dx (1.10.2) 


c+n 


12 Strutt, M. J. O., 1934: Berechnung des hochfrequenten Feldes einer Kreiszylinderspule in einer konzen- 
trischen leitenden Schirmhille mit ebenen Deckeln. Hochfrequenztechn. Elecktroak., 43, 121-123. 
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If, however, the expression does not tend to a limit as € and 7 tend to zero independently, 
it may still happen that 


e0 


c—e b 
lim tf f(a) dx + S f(x) is} (1.10.3) 


exists. When this is the case, we call this limit the Cauchy principal value of the improper 
integral and denote it by 


PV f ræ dx. (1.10.4) 


Finally, if f(x) becomes infinite at an endpoint, say a, of the range of integration, we say 
that f(a) is integrable over a < x < b if 


b 
lim J f(x) dx (1.10.5) 


e>0t 


exists. 
e Example 1.10.1 


Consider the integral fo dx/x. This integral does not exist in the ordinary sense 
because of the strong singularity at the origin. However, the integral would exist if 


d 2 dx 


; x : 
lim — + lim — (1.10.6) 
e>0J_, T 60 Jg £ 


existed and had a unique value as € and 6 independently approach zero. Because this limit 
equals 
lim_[In(e) + In(2) — In(6)] = lim _[In(2) — In(d/e)], (1.10.7) 
€,60 €,6>0 
our integral would have the value of In(2) if 6 = e. This particular limit is the Cauchy 
principal value of the improper integral, which we express as 


pv f E O: (1.10.8) 


We can extend these ideas to complex integrals used to determine the value or prin- 
cipal value of an improper integral by Cauchy’s residue theorem when the integrand has a 
singularity on the contour of integration. We avoid this difficulty by deleting from the area 
within the contour, that portion which also lies within a small circle |z — c| = €, and then 
integrating around the boundary of the remaining region. This process is called indenting 
the contour. 

The integral around the indented contour is calculated by the theorem of residues 
and then the radius of each indentation is made to tend to zero. This process gives the 
Cauchy principal value of the improper integral. The details of this method are shown in 
the following examples. 
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C Ci 
s 
ef 
STs ee ER eM 
-R a a R 


Figure 1.10.1: Contour C used in Example 1.10.2. 
e Example 1.10.2 


Let us show that 


py f SN, aaa (1.10.9) 
oo 8 — r? a 
Consider the integral 
$ i (1.10.10) 
caz 


where the closed contour C consists of the real axis from —R to R and a semicircle in the 

upper half of the z-plane where this segment is its diameter. See Figure 1.10.1. Because 

the integrand has poles at z = +a, which lie on this contour, we modify C by making an 

indentation of radius € at a and another of radius 7 at —a. The integrand is now analytic 

within and on C and Equation 1.10.10 equals zero by the Cauchy-Goursat theorem. 
Evaluating each part of the integral, Equation 1.10.10, we have that 


T pik cos(@)—R sin(0) 5 et? et? 
5 i Re” db + / dz 4 J dz 
y a2 — R2e29% C1 a2 — z2 Cə a2 — z2 


a-n oit a—e eit R eit 
+f 5 3 dx 4 J 5 5 dx 4 i 3 3 dx = 0, (1.10.11) 
-R af — T —atn a — 2x ae & — 2 


where C and C2 denote the integrals around the indentations at a and —a, respectively. 
The modulus of the first term on the left side of Equation 1.10.11 is less than tR/(R? —a?), 
so this term tends to zero as R — oo. To evaluate C1, we observe that z = a+ ee” along 
C1, where 0 decreases from 7 to 0. Hence, 


ee i 0 ciet 
>, dz = li ja + ice” - - d0 1.10.12 
E az” tim, f exp (ia + tee”) — Paced — e2e20i ( ) 
w ; ; pia 

= 43 >. Anh oi i mie 

= lim : exp (ia + ice”) Fa 4 cell d0 = pac (1.10.13) 
Similarly, 
UZ —ta 
f oazie- (1.10.14) 
Co A — % 2a 


as 7 tends to zero. 
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Figure 1.10.2: Contour C used in Example 1.10.3. 


Upon letting R > co, € > 0, and 7 > 0, we find that 


9 Tels Ti ia ian 7 Sin(a) 
ae pug @=-3a, (e — e") = a (1.10.15) 


Finally, equating the real and imaginary parts, we obtain 


py f Sou) 555 He) py f sin(t) geo. (1.10.16) 
9 a — x? a oo a? — x? 
e Example 1.10.3 
Let us show that eae 
/ me) ple (1.10.17) 
bas T aD 
Consider the integral 
f f dz, (1.10.18) 
Cf 


where the closed contour C consists of the real axis from —R to R and a semicircle in the 
upper half of the z-plane where this segment is its diameter. Because the integrand has a 
pole at z = 0, which lies on the contour, we modify C by making an indentation of radius 
eat z =0. See Figure 1.10.2. Because e’*/z is analytic along C, 


Te , =é eit et? R eit 
| etRcos(0)—Rsin(6); qo +J — dx +f —dz+ J — dr =0. (1.10.19) 
0 -R T C * e T 
Since e~ 2 sin) < eF? for 0 <6 <7, 
T iRcos(6)—Rein(0), T LRO Tae 
e id6| < e" db = ———, (1.10.20) 
0 0 R 


which tends to zero as R — oo. Therefore, 


J í ax+ | «dy = f © dz. (1.10.21) 
“o t ce T O * 
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iz d 
[ Se-f Zaif dz- f a E (1.10.22) 
C 7 Cı Z on c 2 


in the limit € > 0 because z = ee”. Consequently, in the limit of € > 0, 


Now 


py f de =n. (1.10.23) 


be 


Upon separating the real and imaginary parts, we obtain 


aa CPSU). ie iy i se EN (1.10.24) 


ax x 


—oCo 


Problems 


1. Noting that 


0—e 
m dp __1_ , fsn@+9)] 
o  cos(y) —cos(@) —sin(@) | sin [$(9 —»)] r , 
and 7 
a dp __1 „i [3(9 +») 
p+ cos(y) — cos(6)  sin(0) | sin [4(0 — y)] ae 
show that a 5 
p 
PV —— = 0 0<90 í 
o cos(y) — cos(8) f ae 
2. Show that 


T. cos(rx/2) TA 


x? —1 


Step 1: Show that 


oo œ pingz/2 
i cee 2) dx = fev | ; i da! ; 


xz? —1 x? 


—oCo 


Step 2: Consider now the integral fọ e'"*/? dz/(z? —1), where the closed contour C consists 
of the real axis from —R to R plus a semicircle of radius R in the upper half of the z-plane. 
See Figure 1.10.1. Because the integrand has poles at z = +1 which lie on the contour, we 
modify C by making an indentation of radius 7 above z = —1 and another indentation of 
radius € above z = 1. Why is this closed contour integral equal to zero? 


Step 3: Show that this contour integral is given by 


i T pin cos(0)/2—Rr sin(@) /2 = J D eit2/2 P eit2/2 P 
Ree 0 R220 — 1 n i Ci 22 -1 aa A z2 — 1 ig 
—1-n eit /2 Te eiT2/2 R eit /2 
+ lim dx+ lim dx + lim dx = 0, 


n-0 _R x2 =I €,n-0 —1+n x2 Sal e>0 1te x2 S 
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where Cı and Ch denote the integrals around the indentations at —1 and 1, respectively. 
Step 4: Show that the first term on the left side tends to zero as R > oo. Why? 
Step 5: Taking z = —1 +e” along C1, where 6 decreases from 7 to 0, show that 


dz = lim - - 
; z2 — n0 J, —2ne% + nre20i 


| eit2/2 ° exp [in(—1 + ne) /2] ine? do = T 
= 
c 


Step 6: Similarly, show that along C2, 


inz/2 0 e im (1 + Gi) /9 , 
J EE ao ip [OEA una 
ca 22-1 630 J, Phe + cs 2 


Step 7: Using Steps 1 through 6, obtain the final result. 


3. Show that 


Co pax _ pbx 
/ << de = nlcot(ar) — cot(bm)},  0<a,b<1. 


Step 1: Consider the integral fo (e%*—e>*) dz/(1—e*), where the closed contour C consists of 
the rectangular box with vertices at (—R, 0), (R,0), (—R,7) and (R, 7), and a semicircular 
indentation Ce at the origin. Show that this closed integral equals zero. Why? 


Step 2: Show that this closed integral may be rewritten, 


—R+ri et? — eb =R et? — eb TE paz _ eo 
lim —— dz+ lim ——dz+ lim — dz 
R-0oo R+qi 1 — e? R=>œ —-R+ri 1 — e? R>œ,>0 J_R 1 — e? 
bz R paz bz R+ri „az bz 
et — e . e =e i É =E 
+ —— dz + lim —— dz + lim ———— dz=0. 

a l-e@ R>œ,>0 Je l-e? R>œ JR 1? 

Step 3: Show that 
Ener i 01 + aee” + a?eee? ik 1 ț beet — bee? 2., % aA 
gee amia =f eee = eT Saas ie 
l-e 30 1-1l-ee 7622/2 
Ce T 
Step 4: Show that 
=R et — eb 0 eth eayt _ e—bR ebyt 

lim —— dz= lim ae idy = 0, 

Raw prar Lae Roo Jy 1— eter 
and : 

R+ri et? — e? T eaReayt _ ebRebyi 
lim ——dz= lim Ban idy = 0, 
Ro JR 1 — e” Roc Jo 1 — etev’ 


if0<a,b<1. 
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Step 5: Show that 
CO par ba CO pax pani co pbx (bri 
CS ee eve 
2 ~ dr= d d 
D 1 — e? - eee K | ve . 


ari metti 


= a “ann = q |cot(am) + i] — r [cot(br) + i]. 


4. Showt? that 


°° 1— cos|[2a(x + ¢)] ee on ree on 
te (x + ¢)? (a? + a?) nae a(¢? + a2)? {2aa(c* + a?) + (C ) 


— 7242 [(¢? — a?) cos(2a¢) + 2a¢ sin(2a¢)] }, 


where a, a, and ¢ are real. 


Step 1: Show that 


(L. AE) f(z) dz = 2ri Res|f (2); ia], 


where Cæ denotes the semicircular contour of infinite radius, Ce is the semicircular inden- 
tation above z = —¢ and 
f= e2ia(z+¢) 


ACE OCEE OS 


Step 2: Taking the limit of R > œ and e > 0, show that 
J f(x) dx = 2ri Res| f(z); ia] + ri Res|f (z); —C]. 


Step 3: Show that 


Ç? — a? — e227 (C? — a?) cos(2a¢) + 2a¢ sin(2a¢)| 
2ia(¢? + a2)? 
Qac + e~24%[(C? — a?) sin(2ac) — 2a¢ cos(2a¢)| 
2a(¢? + a2)? 


Res[ f (z); ia] = 


Res[f(z);—¢] = lim 


z=>—ĻĠ dz 


d [1 — e?e) = 2ia 
z2 + a2 z ¢ + a2 : 


Step 4: Use the results from Steps 1 through 3 to obtain the desired result. 


5. Show that 


py | cat) dx = —r sin(ma), and py [ enn) dx = mcos(ma), 


tT—a 


13 Ko, S. H., and A. H. Nuttall, 1991: Analytical evaluation of flush-mounted hydrophone array response 
to the Corcos turbulent wall pressure spectrum. J. Acoust. Soc. Am., 90, 579-588. 
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where m > 0 and a is real. 


Step 1: Using the complex function e’”*/(z—a) and a closed contour similar to that shown 
in Figure 1.10.2, show that 


a—e R eimz 
ee, 
Coo -R Ce ate} Z7 
Why? Here Cæ denotes the semicircular contour of infinite radius and Ce is the semicircular 
indentation above z = a. 


z=0. 


Step 2: Taking the limit of R > œ and e > 0, show that 


co elma elmz r 
PV dx = ti Res | ——;a| = mie’”™® 
L-a z—a 


—oo 


Step 3: Complete the derivation by taking the real and imaginary parts of the equation in 
Step 2. 


6. Show that 


Rs pemi So gma ni ; ; 
PV = 7i P — 3mi mi 
| pt Ti, and Vf opera” z(e e ), 


where m > 0. 


Step 1: Show that 


oe Lhe T 


Why? Here Cœ denotes the semicircular contour of infinite radius and Ce, and Ce, are 
semicircular indentations above z = —am and z = 7. 


Step 2: Taking the limit of R > œ and e > 0, show that 


© get? ze ze 
PV ag 42 = ti Res | {53-7 | + mri Res |, — 5 
22-7 2 2 


—oco 


= imie ™ + smie™ = īri. 


Step 3: To prove the second relationship, show that 


(Ramee 


Why? Here Cœ denotes the semicircular contour of infinite radius and Ce, and Ce, are 
semicircular indentations above z = 1 and z = 3. 


Step 4: Taking the limit of R > œ and e > 0, show that 


Pv ayer? a a" | rikes| yey 


3mi = Ti (eam _ a) . 


33 


= — lme 11; 
= — zme + zme 
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7. Redo Example 1.10.3, except the contour is now a rectangle with vertices at +R and 
+R + Ri indented at the origin. 


Step 1: Show that along the left side, 


ae el R en’ R 1 A 
— dz| < dy < iE e Ydy=—(l-e“), 
-R z o y R2+y? 0 R ( ) 


which tends to zero as R —- oo. 


Step 2: Show that along the top, 


R+Ri „iz 
€ 
— dz 
R zZ 


which also tends to zero as R + co. Why? 


Step 3: Show that along the right side, 


R+Ri et? 
i — dz 
—R+Ri 7 


which tends to zero as R > co. Why? 


< 2e 


R 
dx 
—R —R 
< ao 20a tee 
[ VR? +x? ( 


Step 4: Just as in the case of the semicircle close contour, we only have an integration along 
the real axis. Do this to complete the problem. 


8. Let us show!* that 


1 >1, 
G(a) = py f as lol 


ar 
Soa a / 2—1’ 
—ı (xz +a)v1 -— z? i lal 0, 


la| <1. 


Step 1: Using the transformation 2iz = z — z~!, show that 


241 sete 
Re a, 1-at=3 (242) 


2 , 


5 Ify +l 1 1 1 (27+ 2iaz-—1 
1— zr? = , and r+a=—[2z a= > ; 
2 z 2i Z 2i z 


Substitute these results into the original integral to find G(q@) as a contour integration on 
the unit circle. 


Step 2: For |a| < 1, we have two singularities within the contours located at z = +V/1 — a?— 
ai. In that case, show that G(a) = 0. 


Step 8: If a > 1, there is a single singularity within the contours and it is located at 


z = iva? —1— ai. Show that G(a) = 1/Va? — 1. 


14 Ott, E., T. M. Antonsen, and R. V. Lovelace, 1977: Theory of foil-less diode generation of intense 
relativistic electron beams. Phys. Fluids, 20, 1180-1184. 
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Step 4: Finally, if a < —1, there is a single singularity within the contours and it is located 
at z = —iVa? — 1 — ai. Show that G(a) = -r /Va? — 1. 


9. Let the function f(z) possess a simple pole with a residue Res[ f(z); c] on a simply closed 
contour C. If C is indented at c, show that the integral of f(z) around the indentation 
tends to —Res[f(z); clai as the radius of the indentation tends to zero, a being the internal 
angle between the two parts of C meeting at c. 


1.11 CONFORMAL MAPPING 


Conformal mapping is a powerful technique for finding solutions, or for simplifying 
the process of finding solutions, to Laplace’s differential equation in two dimensions. This 
method involves introducing two complex variables: z = x + iy and T = p + iø. These 
two complex variables are related to each other via the mapping z = f(r). Under this 
mapping the Argand diagram for the z-variable is mapped into one for the t-variable. In 
certain cases, for example T = yZ, the complex z-plane may only map into a portion of the 
T-plane. In other cases, say T = z + 3i, the complete z-plane would be mapped into the 
complete 7-plane. 

Once we map the original domain into a simpler geometry (a half-plane, circle or 
square), how do we find the solution? There are several techniques available. One method, 
for example, recalls that the real and imaginary parts of an analytic function satisfy 
Laplace’s equation. Therefore, if we could construct an analytic function whose real or 
imaginary parts satisfy the boundary conditions in the new domain, we would have the 
solution in the 7-plane. Then we could use the transformation to obtain the solution in the 
original z-plane. 

What types of functions f(z) are useful? Consider an arbitrary point zo in the complex 
z-plane. Assuming that f’(zo) Æ 0, a straightforward transformation yields 


u u j 2 [0v Ov 
(z2 + a), CO (Sa + a) ie 


where u(x, y) and u(p,c) are solutions to Laplace’s equation in the z and 7 planes, respec- 
tively. Thus, f(z) must be analytic. 


e Example 1.11.1 


In their study of magnetic recording, Curland and Judy!® modeled the ring heads as 
two semi-infinite regions located below the x-axis and running to the right of x = a/2 and 
to the left of x = —a/2. See Figure 1.11.1. 

From symmetry we need only consider the half-space x > 0. Consequently, the new 
boundary consists of the four line segments: AB, BC, CD and DE. If we require that 
the point D in the 7-plane lies at 7 = 1, we shall show in Example 1.11.7 that the desired 
conformal mapping is 


z= < vrai 5 log(— + = (1.11.2) 


15 Curland, N., and J. H. Judy, 1986: Calculation of exact ring head fields using conformal mapping. 
IEEE Trans. Magnet., MAG-22, 1901-1903. 
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ta Z-plane 
a/2 
B C 
to t—plane 
A BC D E 
G ZA ae 


Figure 1.11.1: The conformal mapping used to find the fields of a semi-infinite ring head with a finite gap 
of width a. The potential on the right pole face equals 1 while the potential of the left pole face equals —1. 
In the z-plane the point A is located at (0,00) while point B is located at (0, —00). Because of symmetry 
the potential along the center of the gap AB equals 0. 


A useful method for illustrating this conformal mapping is to draw lines of constant p and 
a in the z-plane. See Figure 1.11.2. This figure shows the local orthogonality between lines 
of constant p and o. 

The greatest difficulty in creating this figure was computing 7 for a given z. This was 
done using the Newton-Raphson method. Starting at the top of the domain, the first guess 
there was given by T = 1 + 7?z?. Marching downward, the T from the previous grid point 
was used for the initial guess. The corresponding MATLAB script is as follows: 


clear; delta = 0.01; % resolution of the grid 


for jj = 1:201 

for ii 1:201 
XX(jj,ii) = delta*ii; YY(jj,ii) = delta*(jj-101); 
RHO(jj,ii) = NaN; SIGMA(jj,ii) = NaN; 

end; end 


% code for the domain x,y >0 


for jj = 1:100 
y = 1 - delta*(jj-1); 
for ii = 1:201 
x = delta*ii; z = complex(x,y); 
if (jj == 1) tau = 1+pixpi*z*z; else tau = TAU(ii); end 
for icount = 1:10 
tempi = sqrt(tau-1); 
temp2 = temp1 - 0.5*i*xlog(1-i*temp1) + 0.5*i*log(1+i*temp1) ; 
ff = temp2/pi + 0.5 - z; deriv = temp! /(2*pi*tau) ; 
temp3 = ff/deriv; tau = tau - temp3; % Newton-Raphson method 
end 
TAU(ii) = tau; RHO(202-jj,ii) = real(tau); 
SIGMA(202-jj,ii) = imag(tau) ; 
end; end 
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Figure 1.11.2: Lines of constant p (dashed lines) and ø (solid lines) given by the conformal mapping 


expressed by Equation 1.11.2. 


% code for the domain 0<a< $ and y <0 
for jj = 1:101 

y = delta - delta*jj; 
for ii = 1:49 

x = delta*ii; z = complex(x,y); 

tau = TAU(ii); % first guess 

for icount = 1:10 

tempi = sqrt(tau-1); 


temp2 = templ - 0.5*i*log(1-i*temp1) + 0.5*i*log(1+i*temp1) ; 
ff = temp2/pi + 0.5 - z; deriv = temp! /(2*pi*tau) ; 
temp3 = ff/deriv; tau = tau - temp3; % Newton-Raphson method 


end 


TAU(ii) = tau; RHO(102-jj,ii) = real(tau); 


SIGMA(102-jj,ii) = imag(tau); 
end; end 


% plot the conformal mapping Equation 1.11.2 


figure 


[C,h] = contour (XX, YY,SIGMA, [0.002,0.05,0.5,5,10,20,30],’k’); 
clabel(C,h,’FontSize’ ,10,’Color’,’k’,’Rotation’ ,0) 
xlabel(’x/a’,’FontSize’ ,20); ylabel(’y/a’, ’FontSize’ ,20) ; 


hold on 
v = [-8,-4,0,4,10,20,30]; 
[C,h] = contour (XX,YY,RHO,v,’--b’); 


clabel(C,h, ’FontSize’,10,’Color’,’b’,’Rotation’ ,0O) 
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x/a 


Figure 1.11.3: The solution to Laplace’s equation when the left boundary is held at 0 while the left and 
top sides of the shaded rectangle are held at 1. This figure shows only a portion of the domain x > 0 and 
ly| < 00. 


Now that we can transform between the z-plane and the 7-plane, and vice versa, let us 
turn our attention to finding the solution to Laplace’s equation in the 7-plane. There the 
solution equals 1 for p > 0 and 0 for p < 0 along o = 0. 

Consider now the analytic function (except at the branch point 7 = 0) 


f(r) =4—log(r)/n. (1.11.3) 


A quick check (using tT = ret?) shows that the imaginary part of f(r), v(r,@) = 1—0/n, 
satisfies Laplace’s equation and the boundary conditions. Thus, constructing the solution is 
as follows: For a given z and y, we use our MATLAB code to compute T. Substituting that 
T into Equation 1.11.3 we compute f(r). Taking the imaginary part, we have the solution 
at x and y. Figure 1.11.3 illustrates the solution for the domain 0 < x < 2 and -1 < y < 1. 


In summary, conformal mapping allowed us to transform the original domain into one 
(an upper half-plane) where we could construct another analytic function whose imaginary 
part satisfied Laplace’s equation and the boundary conditions. A natural question is what 
do we do if we cannot find this analytic function in the r-plane? The next example shows 
an alternative approach. 


e Example 1.11.2 


For our second example of conformal mapping, consider T = Vz? +a?. To illustrate 
this mapping we have constructed two Argand diagrams; one is for the z-plane while the 
second is for the r-plane. Figure 1.11.4 shows how a particular boundary in the z-plane 
maps into the 7-plane. The advantage here is that the infinitely thin filament or peg located 
at z = 0 is completely eliminated in the 7-plane. 

One source of concern is the presence of the square root; for any value of z we would 
have two possible solutions. We make the mapping unique by requiring that S(r) > 0. 
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Figure 1.11.4: The conformal mapping between the z-plane and r-plane achieved by the conformal map- 


ping T = Vz? + a2. 


To better understand this transformation, Figure 1.11.5 illustrates various lines of 
constant R(T/a) and S(r/a) as a function of x/a and y/a. This figure was constructed 
using the MATLAB code: 


clear; 
% compute 7 for various values of z 


for jj = 1:40 
y = 0.05 * jj; 
for ii = 1:42 
x = 0.05 * (ii1-21.5); z = x + ity; tau(ii,jj) = sqrt(z*zta*a) ; 
if (imag(tau(ii,jj)) <= 0) tau(ii,jj) = -tau(ii,jj); end 
X(ii,jj) = x; YGi,jj) = y; 
IM(ii,jj) = imag(tau(ii,jj)); REAL(ii,jj) = real(tau(ii,jj)); 
end; end 


% plot the conformal mapping Equation T = Vz? + a? 


figure 

[C,h] = contour(X,Y,IM,[0.1,0.25,0.5,0.75,1,1.5,2],’k’); 
clabel(C,’FontSize’ ,10,’Color’,’k’,’Rotation’ ,0) 
xlabel(’x’,’FontSize’,20); ylabel(’y’,’FontSize’ ,20); 
hold on 

v = [-1,-0.5,-0.25,-0.01,0.01,0.25,0.5,1]; 

[C,h] = contour(X,Y,REAL,v,’--b’); 
clabel(C,’manual’,’FontSize’,10,’Color’,’b’,’Rotation’ ,0) 


As y > œ, lines of constant S(7/a) become parallel to the boundary y = 0. Only for 
smaller values of y, and as we approach the peg at x = 0, do these lines deviate strongly 
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Figure 1.11.5: Lines of constant R(T/a) (dashed line) and S(7/a) (solid lines) as a function of x and y 
for the conformal mapping T = Vz? + a2. 


from the horizontal as they pass over the obstacle. The smaller the value of S(7/a) the 
more they conform to the shape of the obstacle. 

The behavior of lines of constant R(T/a) are more difficult to understand. There are 
two general classes, depending upon whether the absolute value of R(7/a) is less or greater 
than 1. When |R(T/a)| > 1 they are clearly orthogonal to constant lines of S(7/a). Positive 
values of R(T/a) exist for x > 0 while negative values occur when æ < 0. |R(T/a)| < 1 for 
yea. 

This example has two interesting aspects to it. The first is the presence of the square 
root. The second involves how we will find the solution to Laplace’s equation in the 7-plane. 

Let us assume that in the original z-plane the solution equals zero along the entire 
boundary except along the “peg.” There, the solution equals 1. In the 7-plane the solution 
equals zero along the entire boundary except for the segment —a < p < a, where o = 0, 
along which the solution equals 1. Instead of finding an analytic function whose real or 
imaginary part satisfies this boundary condition, we employ Poisson’s integral formulat? 
for the half-plane y > 0 or Schwarz integral formula:!" 


ulz, y) = : T yf dt. 


TJ œ (2 —t)? +y? 


In the present case, we find that 


u(p,o) = S zy fe ape ue (1.11.4) 


= - fan (22) + tani (S42) (1.11.5) 


16 Poisson, S. D., 1823: Suite du mémoire sur les intégrales définies et sur la sommation des séries. J. 
École Polytech., 19, 404-509. See pg. 462. 


17 Schwarz, H. A., 1870: Über die Integration der partiellen Differentialgleichung 0?u/dx? +0?u/dy? = 0 
fiir die Flache eines Kreises. Vierteljahrsschr. Naturforsch. Ges. Ztirich, 15, 113-128. 
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Figure 1.11.6: The solution of Laplace’s equation when the solution (potential) along the boundary equals 
zero except along the peg located at x = 0. There the solution (potential) equals one. 


Given Equation 1.11.5 we can compute the solution as follows: For a specific value of 
x and y, we find the corresponding value of p and ø. Equation 1.11.5 gives us the solution 
to Laplace’s equation at that point and the corresponding x and y. The MATLAB code is: 


clear; a = 1; 


for jj = 1:100 
y = 0.02 * jj; 
for ii = 1:202 
x = 0.02 * (ii1-101.5); z = x + i*y; tau = sqrt (z*ztaxa) ; 
if (imag(tau) <= 0) tau = -tau; end 
sigma = imag(tau); rho = real(tau); 
X(ii,jj) = x; YGi,jj) = y; 
% Equation 1.11.5 
arg1 = (a-rho)/sigma; arg2 = (atrho)/sigma; 
T(ii,jj) = (atan(argl)+atan(arg2)) / pi; 
end; end 


% plot the solution to Laplace’s equation 


figure 

[C,h] = contourf(X,Y,T,[0,0.05,0.2,0.4,0.6,0.8],’k’); 
colormap autumn 

clabel(C,’FontSize’ ,10,’Color’,’k’,’Rotation’ ,0) 
xlabel(’x’,’FontSize’ ,20); ylabel(’y’,’FontSize’ ,20) ; 


Figure 1.11.6 illustrates this solution. 


So far we have not presented a strategy for finding our conformal mappings. One 
method would be to simply experiment with transforms that had been used in similar 
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problems. Fortunately, during the 1860s, two German mathematicians, E. B. Christoffel!® 
(1829-1900) and H. A. Schwarz! (1843-1921), developed a very popular method of mapping 
a polygon into a half plane. Example 1.11.1 illustrated one of their transforms. Indeed, if 
we imagine that the boundary of the polygon is constructed from a thin wire, the purpose 
of the Schwarz-Christoffel transformation is to unbend the corners so that the wire becomes 
straight. 

Our derivation begins by considering a mapping z = f(r) where 


dz 

qa Pa a) a) (1.11.6) 
and p1, /2,.-.,Pn are any n points arranged in order along the real axis in the 7-plane such 
that pı < p2 < ... < pn. Here the k;’s are real constants and C is a real or complex 


constant. By taking the logarithm of both sides of Equation 1.11.6 we find that 
dz 
log ail = log(C) + ky log(r — p1) + ka log(T — p2) + -+ + kn log(T — pn). (1.11.7) 


We have assumed that the principal value?° of each logarithm is taken. The local magnifi- 
cation factor of the mapping from the 7-plane to the z-plane equals dz/dr, while the angle 
of dz/dr gives the angle through which a small portion of the mapped curve in the 7-plane 
is rotated by the mapping. This angle is given by 


(Z) = 4 (C) + k14 (T — pi) + k24 (T — po) +: + kná (T — pn). (1.11.8) 


dr 


Equation 1.11.8 follows by first taking the imaginary part of Equation 1.11.7 and then 
noting that 4(C) = S[log(C)]. 

Let the point (p,a) = (—oo,0) in the 7-plane be mapped into the point z* in the 
z-plane. See Figure 1.11.7. If we consider the image of a point p as it moves to the right 
along the negative real axis in the 7-plane, then all of the p — p; are real and negative as 
long as p < pı. Hence the angles for all of the p— p; are constant and equal to m in Equation 
1.11.8. Therefore, this equation simplifies to 


6(Z) = 4O) + (i tha tt had (1.11.9) 


Thus the portion of the p axis to the left of the point pı is mapped into a straight line 
segment, making the angle defined by Equation 1.11.9 with the real axis in the z-plane, and 
extending from z* to zı the image of p — p1. 

Now as the point p crosses the point pı on the real axis, the real number p— pı becomes 
positive so that its angle abruptly changes from z to 0. Hence 4 (dz/dr) abruptly decreases 
by an amount kır and then remains constant as T travels from pı to p2. It follows that 


18 Christoffel, E. B., 1868: Sul problema delle temperature stazionarie e la rappresentazione di una data 
superficie. Ann. Mat. Pura Appl., Series 2, 1, 89-103; Christoffel, E. B., 1870: Sopra un problema 
proposto da Dirichlet. Ann. Mat. Pura Appl., Series 2, 4, 1-9. 


19 Schwarz, H. A., 1868: Uber einige Abbildungsaufgaben. J. Reine Angew. Math., 70, 105-120. 


20 For the complex number z = re®, r # 0, the principal value of the logarithm is log(z) = In(r) + 6i, 
where 0 must lie between 0 and 27. 
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Figure 1.11.7: Diagram used in the derivation of the Schwarz-Christoffel method. 


the image of the segment (p12) in the z-plane makes an angle of —ki7 with the segment 
(a 21). 

Proceeding in this way, we see that each segment (fn, Pn+41) is mapped into a line 
segment (Zn, Zn+1) in the z-plane, making the angle of —k,,7 with the segment previously 
mapped. Thus, if the interior angle of the resultant polynomial contour at the point zy, is 
to have the magnitude an, we must set 7 — ap, = —ky7, or kn = A/a — 1 in Equation 
1.11.6. After an integration, we then conclude that the mapping 


= 

—_ of (7 — p1)"*(n — po)" +++ (n — pn) dn + K, (1.11.10) 
where the arbitrary complex constants C and K map the real axis ø = 0 of the 7-plane into 
a polynomial boundary in the z-plane in such a way that the vertices 21, Z2,...,2, with 
interior angles a1, @2,...,@n are the images of the points p1, p2,.-.-, Pn- 


For the final segment T — p > pn the numbers 7 — p; are all real, positive, and equal to 
zero, so that this segment is rotated through the angle 


4(dz/dr) = 4(C), P> Pn- (1.11.11) 
For a closed polynomial the sum of the interior angles is 
ai +a2 +: + an = (n — 2)r. (1.11.12) 


Therefore, 


—2 
ky tho +- + kn = (n )m n= —2. (1.11.13) 
T 


Thus, according to Equations 1.11.8 and 1.11.11, the two infinite segments of the line o = 0 
are rotated through the angle 4 (C) — 27 and 4(C), as is clearly necessary for a closed 
figure. 

What roles do C and K play? Because C is often complex, this constant introduces 
any necessary magnification and rotation of the transformation so that any prescribed 
polynomial in the z-plane is made to correspond point by point to the real axis ø = 0 in 
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Figure 1.11.8: The complex z- and -planes used in Example 1.11.4. 


the 7-plane. In fact, this correspondence can be set up in infinitely many ways, in that 
three of the numbers p1, p2,- .., Pn can be determined arbitrarily. Finally, the mapping can 
be shown to establish a one-to-one correspondence between points in the interior of the 
polygon in the z-plane and points in the upper half of the 7-plane. 


e Example 1.11.3 


Let us derive the conformal mapping used in Example 1.11.2. Referring back to Figure 
1.11.4, we see that ay = 7/2, kı = —1/2, and pı = —a at 21 = 07; ag = 27, k2 = 1, and 
p2 = 0 at z2 = ai; and a3 = 7/2, k = —1/2, and p3 = a at z3 = 0*. Therefore, from 
Equation 1.11.6, 

2 


dz 
Imie 


dt 


Integrating this differential equation, 


z2=CVr?-a+K. (1.11.15) 


Because the point pı = —a corresponds to z = 07, K = 0. Similarly, at p2 = 0, we have 
that 


= Olr +a)? r(r — a)? =0 (1.11.14) 


ai = Cy -a?, or C=1. (1.11.16) 
Therefore, the conformal mapping is given by z = VT? — a2, or T = Vz? + a. 


e Example 1.11.4 


Consider the triangle ABC located in the z-plane as shown on Figure 1.11.8. Here 
we desire to map the interior space of this triangle into the upper half of the 7-plane. At 
point C, points along the boundary and to the left of C are to be mapped out to —co in 
the t-plane while points along the boundary and to the right of C are mapped to +00. 

From Equation 1.11.6 we have that 

dz 


T e pA Orn 1 ig I, (1.11.17) 
r 


Complex Variables 69 


Figure 1.11.9: The complex z- and 7-planes used in Example 1.11.5. 
Integrating this differential equation, 
z 
z= f ieee ne alma dy +H K. (1.11.18) 


Because we want the points 7 = 0 and z = 0 to correspond to each other, K = 0. On the 
other hand, if we wish 7 = 1 and z = 1 to correspond, Equation 1.11.18 yields 


: a/n—1 = B/n-1 = T'(a/r)0(6/r) = 
cf n (-n) dn = OED =1, (1.11.19) 
where T (-) is the gamma function defined by 
T(x) = a te" dt. (1.11.20) 
Consequently, 
tie gi LEENA (1.11.21) 
Pa/m)P(8/m)’ a 
and 
ee T[(& + B)/7] ij a/m—1/4 _ ,,\B/nr-1 
E, n (-n) dn. (1.11.22) 


A noteworthy aspect of this example is that the conformal mapping is given by an integral 
and not some analytic expression. 


e Example 1.11.5 


Consider the domain lying in the upper half of the z-plane except for a triangular section 
BCD shown in Figure 1.11.9. We wish to construct the Schwarz-Christoffel transformation 
that maps this domain into the upper half of the 7-plane. From Equation 1.11.6 we have 
that 

dz 


T =C'(r +1)“ a)/n 17 (7+2a)/7 ‘= vp haa dis (1.11.23) 
T 


ee a (1.11.24) 
= (T2 a 1)2/7 = (1 = T2)a/n* chee 
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Figure 1.11.10: The complex z- and 7-planes used in Example 1.11.6 with a < 1. 


Integrating this differential equation, 


n?e 
z= cf g oan +K. (1.11.25) 


If we want the point 7 = 0 to correspond to the point z = ki, then K = ki. On the other 
hand, if the point 7 = 1 corresponds to z = a, then 


nroln 
a=0 f g” aor t+ hi. (1.11.26) 


Solving for C, 
Ja — ki) 
C= 7 . 
T (a/r + 3) r(1-— a/r) 


(1.11.27) 


Therefore, the final answer is 


i vTla-— ki) T palT , 
a Tr (a/r +4 5) I'(1 — a/r) f (i= n2yol™ dn + ki. (1.11.28) 


e Example 1.11.6 


Consider the domain within the L-shaped boundary shown in Figure 1.11.10. We wish 
to construct the Schwarz-Christoffel transform that maps the interior into the upper half 
of the 7-plane. Note that we broke the boundary in such a manner that points slightly to 
the left of point A are mapped to —co while points slightly below the point A are mapped 
to +00. 

Because a < 1, Equation 1.11.6 gives 


a = O(r +1) P(r +a) Prr- a) P(r- 1). (1.11.29) 
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Figure 1.11.11: The complex z- and T-planes used in Example 1.11.7. 


Integrating this differential equation, 


+K, (14.11.30) 


n-— 1) dn Cus? (n — 1) dn 
2=C K= 
E r wh JE )(1 = p?’n?) 


where p? = 1/a?. To compute C and K, we would need further information. 


e Example 1.11.7 


Let us derive the conformal mapping, Equation 1.11.2, used in Example 1.11.1. The 
z— and 7—planes are shown in Figure 1.11.11. From this figure we see that a; = 37/2, 
ag = 7/2, a3 = 1/2, pı = 1, p2 = 07, and p3 = 07. This yields 


Me K(r — 1) -1r — 9/21 (r — gty™/@n)-1 = KV r (1.11.31) 


dr z 


Integrating Equation 1.11.31, we find that 


; a 1l+iv7r—1 
z= 2K [V7 —1-arctan(V7 —1)] +C =2K [v7 14 toe — |) +C. 
(1.11.32) 
Because at 7 = 1, z = a/2, we have C = a/2. 
The computation of K is more complicated. Referring to Figure 1.11.11, we note that 


C Co! 
VT—-1 
dz= | K v. (1.11.33) 
B B' T 
Setting T = r e°” with r > 0, Equation 1.11.34 becomes 
“= K lim nee Dpat dé = Kr. (1.11.34) 
2 r=>0 re pei 


Thus K = a/(27) and we recover Equation 1.11.2. 
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Problem 5 


Problems 
1. Verify that the function 7 = e” maps the strip 0 < S(z) < m into the half-plane S(r) > 0. 


2. Verify that the function 7? = 1 — e? maps the strip -m < S(z) < m, except for the 
negative real axis, into the upper half of the 7-plane. 


3. Use the Schwarz-Christoffel method to find the conformal mapping that maps the quarter 
plane x > —7, y < 7 into the upper half of the 7-plane. We require that the point (—7, 7) 
in the z-plane maps to the point (0,0) in the r-plane. 


4. Use the Schwarz-Christoffel method to find the conformal mapping that maps the sector 
lying between the z-axis and the line 0 = 7/3 into the upper half of the 7-plane. We require 
that the point (0,0) in the z-plane maps to the point (0,0) in the 7-plane. 


5. Use the Schwarz-Christoffel method to find the conformal mapping that maps the portion 
of the z-plane defined by 0 < r < œ, 0 < 6 < 77/4 into the upper half of the r-plane. We 


Complex Variables 73 


Tt plane 


Bs LG pP,=-1 P2=1 


[A a YU 


Problem 6 
yt to 
z-plane t— plane 
WG 
5 
Z 
Z 
Y 
VAAN Wynn R7 RE Zp 
Problem 7 
y fo 
z-plane T- plane 
ai 
UL 
F A BC OD 
UMMM ABK < yp 
WMI 


Problem 8 


require that the points (0,0) and (1,0) in the z-plane map to the points (0,0) and (1,0) in 
the 7-plane, respectively. 


6. Use the Schwarz-Christoffel method to find the conformal mapping that maps the domain 
|x| < a, 0 < y into the upper half of the 7-plane. Let the point (—a,0) become the point 
(—1,0) while the point (a,0) becomes the point (1,0). 


7. Use the Schwarz-Christoffel method to find the conformal mapping that maps the region 
x >0,0< y< a into the upper half of the 7-plane. We require that the point (0,a) maps 
to (—1,0) in the 7-plane while the point (0,0) maps to (1,0) in the 7-plane. 


8. Use the Schwarz-Christoffel method to find the conformal mapping that maps the region 
shown in the figure into the upper half of the 7-plane. We require that the points (0,0) and 
(0,a) in the z-plane map to the points (0,0) and (1,0) in the 7-plane, respectively. 


9. Construct a transform between a z-plane which has a barrier that runs parallel to the 
z-axis from z = L+aLi to œ +7Li and a -plane that has no barrier. 
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Step 1: Begin by using the Schwarz-Christoffel method to show that the conformal mapping 
pictured in Figure 9a is given by 
dz 


I” Cr! (r —1)*, 


where kı = —a/(27) and kz = a/r — 1. 


Step 2: Next, consider the limit as the points B and D in the z-plane in Figure 9a move 
out to infinity (so that a + 2r) and we obtain Figure 9b. Consequently, the transform 
approaches 


dz T-1 
rr gae 


or 
z = CĪr — log(r)| + K. 


Here we have taken the principal branch of the logarithm so that log(z) = In(|z|) +10 where 
0 < 6 <7. (We do not require that 0 < 0 < 2m because we are always in the upper 
half-plane.) 


Step 3: Following Example 1.11.7, consider the area around 7 = 0. Show that 
dz x age = —iC dé, 
T 


where 7 = re”. Integrating from point Bi to point B}, show that C = L. 


Step 4: To compute K, note that if the point C, located at z = L + mLi, corresponds to 
the point C”, located at 7 = 1, then K = 711. 


10. Use conformal mapping to solve Laplace’s equation for the infinite strip —co < x < a, 
0<y< xr. The solution equals zero everywhere along the boundary except for x > 0, 
y = 0, where u(x, 0) = 1. 


z 


Step 1: Consider the mapping T = e*. Show that p = e*cos(y) and o = e” sin(y). In 
(co, 0) — (00, 0). 


Complex Variables 75 


+> Z-plane 
D 
B, 
A ag 
WEE PE R AGR G RA GA LGA GG AGG, X 


0 t— plane 


/ 
A’ ByE c’ D 


Yo 
il cde EE p 


Problem 9b 


+ Z-plane 


z,=0 yy i, x 


Wi MWwtw@=$t WH ETT 


Figure 1.11.12: The conformal mapping between the z-plane and r-plane achieved by the conformal 


mapping T = gle, 


Step 2: Using Poisson’s integral formula for the upper half-plane, show that 


1 fa _,fl-2 1 zi y 
Seet =l= t =) 
tee ae ee ee) 


Step 3: Show that 


1 Tsj 
u(x, y) = 1 — — tan™! eC ; 
T e” cos(y) — 1 
11. Use conformal mapping to solve Laplace’s equation for a pie-shaped sector in the first 
quadrant. See Figure 1.11.12. The solution equals zero along the entire boundary except 
for 0 < x < 1 where it equals one. 


Step 1: Show that the mapping z = r°/7 or T = 27/% maps the pie-shaped sector into the 
half-plane S(T) > 0. See Figure 1.11.12. 
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Step 2: Using Poisson’s integral formula for the upper half-plane, show that 


Pot =p 
> . 


1 
u(p,o) = E cot" ( 


Step 3: Show that 


T/a _ 
FOO et |! aa . 
T 


sin(70/a) 
where x = rcos(@) and y = r sin(0). 
12. Use conformal mapping to solve Laplace’s equation for the semi-infinite strip 0 < x <a, 
0< y< œ, where u(x,0) =1,0< a <a, and u(0,y) = u(a,y) =0,0<y< œ. 
Step 1: Consider the mapping T = — cos(mz/a). Show that 
p = —cos(ra/a)cosh(ry/a), and o =sin(rx/a)sinh(my/a). 


In particular, (0,00) — (—oo,0), (0,0) > (—1,0), (a/2,0) — (0,0), (a,0) > (1,0), and 
(a, œ) + (00,0). 


Step 2: Using Poisson’s integral formula for the upper half-plane, show that 


pis) 


E 
u(p,o) = = cot q 25 


Step 3: Show that 
u(x, y) = Leonil [sinn (7#) = sin? (75)] 1 sin(“) sin (72) } 
= Z tanm sin() /snn()| i 


Step 4: In the case that boundary conditions read u(0,y) = u(a,y) = 1 for 0 < y < œ 
and u(x,0) = 0 for 0 < x < a, how could you use the solution in Step 3 to solve this new 
problem? 


Further Readings 
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Cambridge University Press, 660 pp. Covers a wide variety of topics, including complex 
numbers, analytic functions, singularities, conformal mapping and the Riemann-Hilbert 
problem. 


Carrier, G. F., M. Krook, and C. E. Pearson, 1966: Functions of a Complex Variable: 
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pp. Classic textbook. 


Flanigan, F. J., 1983: Complex Variables. Dover, 364 pp. A crystal clear exposition and 
emphasis on an intuitive understanding of complex analysis. 
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Chapter 2 
Advanced Transform Methods 


In their course work, most engineering students are introduced to the concept of the 
Fourier and Laplace transforms. The presentations are limited because the student has not 
studied complex variables. Having presented this topic in the previous chapter, the reader 
is ready to deepen his/her ability to use these transform methods. 

This chapter deals with two important aspects of transform methods. In the past you 
may have inverted Fourier and Laplace transforms using partial fractions, tables and some 
general properties of the transform. Often these techniques fail and here we show how the 
power of complex variables can overcome these difficulties. 

The reason that Laplace transforms are taught to engineers is their ability to solve 
ordinary differential equations. When it comes to partial differential equations the student 
is only taught one method: separation of variables. In Sections 2.4 through 2.6 we show 
how Laplace transforms can be used to solve the wave, heat, and Laplace equations. 


2.1 INVERSION OF FOURIER TRANSFORMS BY CONTOUR INTEGRATION 


Although we may find the inverse by direct integration or partial fractions, in many 
instances the Fourier transform does not lend itself to these techniques. On the other hand, 
if we view the inverse Fourier transform as a line integral along the real axis in the complex 
w-plane, then some of the techniques that we developed in Chapter 1 can be applied to this 
problem. To this end, we rewrite the inversion integral for the Fourier transform as 


s) = = i F(w)e dw = > f Flee d- f| Fedr (2.1.1) 


as 20 Cr 
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where C denotes a closed contour consisting of the entire real axis plus a new contour 
Cr that joins the point (co,0) to (—00,0). There are countless possibilities for Cr. For 
example, it could be the loop (co, 0) to (co, R) to (—oo, R) to (—00, 0) with R > 0. However, 
any choice of Cg must be such that we can compute fo, F (z)e"* dz. When we take that 
constraint into account, the number of acceptable contours decreases to just a few. The 
best is given by Jordan’s lemma.' 


Jordan’s lemma: Suppose that, on a circular arc Cr with radius R and center at the 
origin, f(z) — 0 uniformly as R > co. Then 


(1) lim (Ze"™ dz = 0, (m > 0) (2.1.2) 


R- oo Cr 


if Cr lies in the first and/or second quadrant; 


R- oo 


(2) lim f fle dz = 0, (m > 0) (2.1.3) 
CR 
if Cr lies in the third and/or fourth quadrant; 


(3) lim Í f(zje™’* dz =0, (m > 0) (2.1.4) 


R= 


if Cr lies in the second and/or third quadrant; and 


(4) lim J f(z) ™ dz = 0, (m > 0) (2.1.5) 
R-o0o0 Cr 
if Cr lies in the first and/or fourth quadrant. 


Technically, only (1) is actually Jordan’s lemma, while the remaining points are varia- 
tions. 


Proof: We shall prove the first part; the remaining portions follow by analog. We begin by 
noting that 


ree if Hoe” del < J IF) |e] ldz]. (2.1.6) 
CR CR 
Now 
ld2|= Rd, |f(2)| < Mnr, (2.1.7) 
tere = |exp(imRe”)| = |exp{imR[cos(0) + isin(0)]}| = e782, (2.1.8) 
Therefore, 
01 
\IR| < RMr f exp|-mR sin(9)] dé, (2.1.9) 
0 


1 Jordan, C., 1894: Cours D’Analyse de Ecole Polytechnique. Vol. 2. Gauthier-Villars, pp. 285-286. 
See also Whittaker, E. T., and G. N. Watson, 1963: A Course of Modern Analysis. Cambridge University 
Press, p. 115. 


Advanced Transform Methods 79 


where 0 < fo < 0; < m. Because the integrand is positive, the right side of Equation 2.1.9 
is largest if we take 6) = 0 and 0, = a. Then 


T f T/2 , 
\Ipn| < RMR J e ™Rsin®) dg = RMR f e~ ™Rsin(O) g9, (2.1.10) 
0 0 


We cannot evaluate the integrals in Equation 2.1.10 as they stand. However, because 
sin(@) > 20/x if 0 < 0 < 2/2, we can bound the value of the integral by 


m/2 
|Ir| < 2RMp | e72mRO/T ag = Č Mr (1-e"™*). (2.1.11) 
0 m 


If m > 0, |Ip| tends to zero with Mp as R > ow. 
Consider now the following inversions of Fourier transforms: 
e Example 2.1.1 


For our first example we find the inverse for 


1 


F = ,b > 0. 2.1.12 
(a) w2 — 2ibw — a? — b2’ X ( ) 
From the inversion integral, 
1 co etto 
t)h= d 2.1.13 
FO a Me ere ere j ( ) 
or 
1 ettz 1 el 
t)h= d d 2.1.14 
F(t) z f Soo á ae Pie ae ( ) 


where C denotes a closed contour consisting of the entire real axis plus Cr. Because 
f(z) = 1/(2? — 2ibz — a? — b?) tends to zero uniformly as |z| + oo and m = t, the second 
integral in Equation 2.1.14 vanishes by Jordan’s lemma if CR is a semicircle of infinite radius 
in the upper half of the z-plane when t > 0 and a semicircle in the lower half of the z-plane 
when t < 0. 

Next we must find the location and nature of the singularities. They are located at 


z — 2ibz — a? -b =0, or z = +a + bi. (2.1.15) 


Therefore we can rewrite Equation 2.1.14 as 


1 ettz 
t) = dz. 2.1.16 
P(e) zÉ (z—a-— bi)(z +a- bi) ‘ ( ) 
Thus, all of the singularities are simple poles. 
Consider now t > 0. As stated earlier, we close the line integral with an infinite 
semicircle in the upper half-plane. See Figure 2.1.1. Inside this closed contour there are 
two singularities: z = +a + bi. For these poles, 


ett? (z — a — bi)e* 
R ; bi) = li 2.1.17 
e api" t i) srati (z—a-— bi)(z +a-— bi) ( ) 
iat ,—bt —bt 
2e x = Sz leos(at) +isin(at)], (2.1.18) 
a 
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Cr fort<0 


Figure 2.1.1: Contour used to find the inverse of the Fourier transform, Equation 2.1.12. The contour C 
consists of the line integral along the real axis plus Cr. 


where we used Euler’s formula to eliminate e**. Similarly, 


itz —bt 
Res( 3 - ck pine + ui) = ———[cos(at) — isin(at)]. (2.1.19) 


Consequently, the inverse Fourier transform follows from Equation 2.1.16 after applying the 


residue theorem, and equals 
—bt 


f) = a sin(at) (2.1.20) 


for t > 0. 

For t < 0, the semicircle is in the lower half-plane because the contribution from the 
semicircle vanishes as R — oo. Because there are no singularities within the closed contour, 
f(t) =0. Therefore, we can write in general that 


—bt 
f(t) = oe sin(at) H(t). (2.1.21) 


e Example 2.1.2 


Let us find the inverse of the Fourier transform 


—wi 


e 


Flw) = —— 2.1.22 
where a is real and positive. 


From the inversion integral, 


1 œ silt—1)w 1 i(t—1)z 1 i(t—1)z 
f(t) i 2 d fi dz l £ dz, (2.1.23) 


= UW = 
2T J ~ W? +a? Qn Jo z2? +a? 2T Jog 22 +a? 
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where C denotes a closed contour consisting of the entire real axis plus Cr. The contour 
Cr is determined by Jordan’s lemma because 1/(z? + a?) > 0 uniformly as |z| — oo. Since 
m = t — 1, the semicircle Cr of infinite radius lies in the upper half-plane if t > 1 and in 
the lower half-plane if t < 1. Thus, if t > 1, 


1 f ei(t-1)z e7a(t-1) 
f()= z OniyRes| S z zail T. (2.1.24) 
whereas for t < 1, 
1 5 n ei(t-1)z ealt-1) ase 
t I ee Wt ) ; } = . ade 
F(t) = Erie SG ai] = (2.1.25) 


The minus sign in front of the 277 arises from the clockwise direction or negative sense of 
the contour. We can write the inverse as the single expression 


e~alt—1]| 
t) = ; 2.1.26 
te) = (2.1.26) 
e Example 2.1.3 
Let us evaluate the integral 
°° cos(ka) 
1. za tt (2.1.27) 


where a, k > 0. 
We begin by noting that 


®© cos(kx) — œo eike 7 | cikz 
I T2 + a2 =n PES ax) = #( ee 5 (2.1.28) 


where Cı denotes a line integral along the real axis from —oo to oo. A quick check shows 
that the integrand of the right side of Equation 2.1.28 satisfies Jordan’s lemma. Therefore, 


oo elkr eikz eikz 
< dr = § ——~ dz = 2mi Res( ———; ai (2.1.29) 
-œ T? + a? c 2? +a? z2? +a?’ 
PE PE 1 E 
= 2ni lim C = Zeka, (2.1.30) 
zai zł + a2 a 


where C denotes the closed infinite semicircle in the upper half-plane. Taking the real and 
imaginary parts of Equation 2.1.30, 


J cos(ka) de= Tete d Í sin(kx) i (2.1.31) 


2 2 2 2 
et al a œ t +a 
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original 


Figure 2.1.2: Contour used in Example 2.1.32. 
e Example 2.1.4 
Let us now invert the Fourier transform F(w) = 2a/(a? — w?), where a is real. The 


interesting aspect of this problem is the presence of singularities at w = +a that lie along 
the contour of integration. How do we use contour integration to compute 


f(t) = of" ee (2.1.32) 


TJ nw 


The answer to this question involves the concept of Cauchy principal value integrals, 
which allows us to extend the conventional definition of integrals to include integrands 
that become infinite at a finite number of points. See Section 1.10. Thus, by treating 
Equation 2.1.32 as a Cauchy principal value integral, we again convert it into a closed 
contour integration by closing the line integration along the real axis as shown in Figure 
2.1.2. The semicircles at infinity vanish by Jordan’s lemma and 


a ettz 
f= T o a lz; (2.1.33) 
m joa- z 
For t > 0, 
2ria 1 ee 2ria 1 ett? 
f®)= m Rela | = Resla] ; (2.1.34) 


We have the factor ł because we are only passing over the “top” of the singularity at z = a 
and z = —a. Computing the residues and simplifying the results, we obtain f(t) = sin (at). 


Similarly, when t < 0, 


T 2ria 1 


fi) == sRes| 


ett? l 2ria 1 
2) 


Utz 
sRes| aia = ~sin(at). (2.1.35) 


z2—a 
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These results can be collapsed down to the single expression f(t) = sgn(t) sin(at). 


e Example 2.1.5 


An additional benefit of understanding inversion by the residue method is the ability 
to qualitatively anticipate the inverse by knowing the location of the poles of F'(w). This 
intuition is important because many engineering analyses discuss stability and performance 
entirely in terms of the properties of the system’s Fourier transform. In Figure 2.1.3 we 
graphed the location of the poles of F (w) and the corresponding f(t). The student should 
go through the mental exercise of connecting the two pictures. 


e Example 2.1.6 


So far, we used only the first two points of Jordan’s lemma. In this example? we 
illustrate how the remaining two points may be applied. 
Consider the contour integral 


e~ en 
t d 
a E | 
where c > 0 and 8,T are real. Let us evaluate this contour integral where the contour is 


shown in Figure 2.1.4. 
From the residue theorem, 


f cot(z) | o | 2 | dz 
c (T +2r2)? +B? (T-—2rz)? + 8? 


= 2ri DD Res{ eot(r2) lz T a Fa H TE a z z| ; n) 


n=1 
; e e |r| + 62 
+ Qi Res{ cot(r2) Ee Pua G= tees z| Oo \ 
+ 2ni Res? cot(rz) = = ZAN (21.36) 
(T +2rz)? +8? (T-—2rz)}? + p? 27 


Now 


—CzZ 


i , 2 g Oz 2 
{eo me) Ee Fonz? +B! (r 2m2) + =| n} 


as (z T n) cos(1z) P E S$ eo 

= ie sinma iml | Qnz)? B2 T (7 = nz)? i = (2.1.37) 
1 eve enne 

~ lz + 2nr)? + 82 5 (7 — 2n7)? + z| , (2.1.38) 


—CZ 


e e77 „|r| + Be 
Resf cated) lz +272)? +8? (rT—2rz)? + z| ’” 2m \ 


E Te cot(rz)| (z-|r|- bie (z-|r|- Bie? 
m e a 2189) 
z cot(|r|/2 + 8i/2) exp(—c|r|/2r)[cos(cß/2r) — i sin(cß/2r)] (2.1.40) 


Ar Bi 


2 See Hsieh, T. C., and R. Greif, 1972: Theoretical determination of the absorption coefficient and the 
total band absorptance including a specific application to carbon monoxide. Int. J. Heat Mass Transfer, 
15, 1477-1487. 
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Figure 2.1.3: The correspondence between the location of the simple poles of the Fourier transform F'(w) 
and the behavior of f(t). 
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> 


i 2m 

D <x = 
x |TI-L 

i 2T 


Figure 2.1.4: Contour used in Example 2.1.6. 


and 


e e7 l |r| r= Bi 
Res cot(mz) lz mora ean =| x \ 
wo ` cot(7z) (z © |r| + Bije” (z = ir] + Bije” 
= JARS AH ee 4r? iz + T/2r)? + 82/42 T (z — 7/27)? + m (2.1.41) 
_ cot(|T|/2 — Bi/2) exp(—e|r|/27) [cos(e8/2m) + isin(c8/27)] 
p —4rpi : 


(2.1.42) 


Therefore, 


$ cot(ne) | = + = | dz 
c (T +2r2z)? +8? (T — 2nz)? + 8? 


es enone ewe 

‘Slee ae 

x i ell + eb 
28 eżlTl — el 
i ell e? 2 

— aaa a ™cos(cB/2m) + isin(c8/27)] (2.1.43) 


e~ I71/2" (eos(c8/2m) — isin(cB/27)| 


f oo ewe enone 
= AS a +82 (T — 2nr)? a 


i sinh(G) cos(c3/27) + sin(|T|) sin(c3/27) 
B cosh() — cos(r) 


gen, (2.1.44) 


where cot(a) = i(e”* + 1)/(e%% — 1), and we made extensive use of Euler’s formula. 

Let us now evaluate the contour integral by direct integration. The contribution from 
the integration along the semicircle at infinity vanishes according to Jordan’s lemma. In- 
deed, that is why this particular contour was chosen. Therefore, 
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f cot(re) | = + = | dz 
c (T +272)? +8? (T —2rz)? + 8? 
= 1. cot(re) | B H = dz 
z (tone + Rt e TA 
a J cot (rz) | i aes | dz 
C. (T +2r2)? +8? (T — 2nz)? + 8? 


—t00 en ene 
y ie ae li +272)? +82 ` (T — 2r2)? + z| a (2.1.45) 


Now, because z = iy, 


I cot(rz) = + e dz 
P (T+ 2r2z)? +B? (T-—2rz)}? + p? 
a a e d 2.1.4 
= [cot | ae "iG Ong a yY (2.1.46) 


oo h 2 i 4 23,2 \po— icy 
aa coth(zy)(r* + 6 my" je dy, (2.1.47) 
pe CB apts ony? 


oe r ) en z en F 
E ee (T +2r2)? + 6? 9 (7 — 2mz)? + 8? g 
e 


—oco ety 
bi / coth(ny)| Aaa Cae. 


=e 


co th 2 2 4 2,,2\ ptcy 
= SOS oe EE iy (2.1.49) 
e (T2 + B2 — 4n2y?)? + 169272 y? 


—icy 


and 


| cot(re) | = H = dz 
C. (T +272)? +8? (7 — 2rz)? + 8? 


=r /2 1 bi : 
a [ae — | cia 
1/2 Tee?" 3 


exp(—cee™) exp(—cee™) 
H : ; 2.1.50 
z [z + 2ree™)2 + B2 (T — 2ree?t)2 + 82 (2150) 
In the limit of € > 0, 
ETER esz 
t H d 
feo (r)| + 82 (T — 2772)? =| zZ 
© coth 2 1 ae ee 2j 
=4i f Bi AUL PA Sim CT : (2.1.51) 
o (T2 + 62 — An? y?)? + 1602772 y2 T? + 6? 
ad ene enne 
‘Le + 2n7)? + 8? $ (T — 2nT)? + z| 
i sinh(8) cos(c3/27) + sin(|7]) sin(¢B/2m) | —clr|/2m_ (2.1.52) 


B cosh(3) — cos(r) 
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or 


af coth(my) (T? + B? — 4n*y?) sin(cy) 
o (T? Fp- 4n2y2)? + 1602724? 


Og enne enne 
= 2 lz Tina et G mn A (2.1.53) 
1 sinh(8) cos(c8/2m) + sin(|r|) sin(cb/27) art jon 
B cosh(8) — cos(T) < TRE Ge: 
If we let y = «/2r, 
B coth(x/2)(r? + 8? — x?) sin(cx/27) 
ef (7? +B? — 27)? 24772" ay 
ere e 

z 27 | (7 +2nr)? + 8? t (7 — 2nm)? + 8? ape) 

sinh(3) cos(cB/27) + sin(|r|) sin(cG/27) e—elrl/2r 4 26 l 

cosh(8) — cos(r) i 


Problems 


By taking the appropriate closed contour, find the inverse of the following Fourier transforms 
F(w) by contour integration. The parameter a is real and positive. 


1 w 3 w w? 
po ae pte ae ` WwW +a? * (Ww? + a2)? 
1 1 2 3 
5. > 6 —— te = 8. i 
w? — 3iw — 3 (w — ia)2r+2 (w? — 1)? + 4a2w? (2 — wi)(1 + wi) 


Then check your answer using MATLAB. 


9. Find the inverse of F(w) = cos(w)/(w? + a”), a > 0, by first rewriting the transform as 
ew 


Hey 2(w? + a?) T 2(w? + a?) 


and then using the residue theorem on each term. 


10. Find? the inverse Fourier transform for 


exw etiw-iw 
Fy = r p = - 
+ (w) (w = ai) (Rew! E ev?) (w = ai) (R2 = e—2wi)’ 


where a > 0 and R>1. 


3 See Scharstein, R. W., 1992: Transient electromagnetic plane wave reflection from a dielectric slab. 
IEEE Trans. Educ., 35, 170-175. 
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Step 1: Show that 


co i(t—11)w 
fel) = 5 [ E 


~ On oo (w — ai)(R?2 — e727) a 


Step 2: Show that the singularities consist of simple poles at z = ai and zn = nr +iln(R), 
where n = 0, +1, +2, £3,.... 


Step 3: For t > 0 show that 


1 co etto 
H= a Go iR? ey 


“SOQ 


and we must close the contour with an infinite semi-circle in the top half-plane. 


Step 4: Show that 


fo) eitw eit 
- = 2ri —;ai 
/ C= =e o Refy Zai (R? — e727)’ ail 


where y ; 
e zZ eG 
Ri - = 
ce | (z Tes ai) (R? pae e7 22%) ? ail R2 e2a : 
and : 
el Ro-texinnt 
Res n| = > , 
z — ai) (R2 — e~ 27") 2iR? {nr + [In(R) — a] i} 
so that 


oe innt 


= l 5 f ; 
R? — e?a 2Rt+2 nr + [In(R) — ali 
n=— 0 
Step 5: For the case t < 0, show that we close the contour with an infinite semi-circle in 


the bottom half-plane to compute f4(t). 


Step 6: Compute the residues of the enclosed singularities in Step 5 and show that f4 (t) = 0. 
Why? 
Step 7: Show that f(t) equals 


oe innt 


ERE.) e7% H(t) £ 
A@® = R2 — e24 (t) + 2Rt+2 De nm + (In(R) — ali 


at any time t. 


Step 8: For F_(w), show that 


oo i(t—2)w 
f= = f : 


On Joo (w — ai)(R2 — e207) 


Step 9: For t > 2, show that we close the contour with an infinite semi-circle in the top 
half-plane. 
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Step 10: Compute the residue of the enclosed singularities and show that 


oo ei(t-2)w i er ei(t—2)z f 
if wo (w — ai) (R? — em Bet) OA e aye ey 


Bes Sea m ei(t—2)z 
+ 271 2 elz a(R — 24) in : 
where 
A ei(t—2)z ent 
lz — ai) (R2 — e27) ail T Reg’ 
and , i 
RS ei(t—2)z R tt2etinnt 
~; Zn] = = =. 
(z — ai) (R2 — e7?) 2iR? {nr + [In(R) — ali} 
so that 


TFE foe) ; 
ie at 1 innt 


e 
FO = Renit aR 2 nm + [In(R) — a] i` 


Step 11: For t < 2 we must close the contour with an infinite semi-circle in the bottom 
half-plane. Compute the residue of the enclosed singularities and show that f_(t) = 0. 
Why? 


Step 12: Show that the final answer for f_(t) at any time t is 


of innt 


LA) ie tt HG je ea S a 7 


~ Re] 2Rt In(R) — ali 


n=— 


11. During the solution of the heat equation, Taitel et al.4 inverted the Fourier transform 


cosh(yVw? + 1) 
Vw? +1 sinh(pVw? + 1/2)’ 


F(w) = 


where y and p are real. 

Step 1: From the definition of the Fourier transform, show that 
1 cosh(yvVz? + 1) e*# 

— Zy 
2r Jo vz? +1 sinh(pv2? + 1/2) 


where we have closed the line integral with an infinite semicircle in the upper half-plane if 
t > 0. For t < 0 we close the contour in the lower half-plane. 


f(t) 


Step 2: For t > 0, show that the enclosed singularities are simple poles that are located at 
z =i and pvz? +1 = 2n7i,n=0,1,2,..., or Zn = 1/1 + 4n? 77? /p?. 

Step 3: Show that 

cosh(yvVz? + 1) e** J] e~ 


Res 32 ae 
Vz? +1 sinh(pv2z? + 1/2) ip 


4 Taitel, Y., M. Bentwich, and A. Tamir, 1973: Effects of upstream and downstream boundary conditions 
on heat (mass) transfer with axial diffusion. Int. J. Heat Mass Transfer, 16, 359-369. 
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and 
k ee EDA izt 2cos(2nry/p)exp(— y 1 + 4n?r?/p2 t) 
ES : 
Vz? +1 a a 1/2)’ ip(—1)"./1 + 4n2n?/p? 


Step 4: For t > 0, show that 


= ” cos(2nmy/p) exp 1 + 4n?r?/p?t 
ie /p) p mi) 


y1 + 4n?r?/p? 


Step 5: For t < 0, show that the enclosed singularities are simple poles located at z = —i 
and zn = —i\/1 + 4n?r?/p?. 


Step 6: Show that 


i | cosh(yvz? +1) et i et 
es ye SS 
V2? +1 sinh(pV2? + 1/2) ip 


and 


x — cosh(yv/z? + ke ew 2cos(2n7y/p) exp(4/1 + 4n?7?/p? t) 
es l 


in| = 
Vz? +1 sinh(pV2? + 1 1/2) ip(—1)"VJ/1 + 4n? r? /p? 


Step 7: For t < 0, show that 


t 


2 3 (—1)” cos(2n7y/p) exp(./1 + 4n?7? /p? t) 


y1 + 4n?r?/p? 


Step 8: Show that we write the results from Step 4 and Step 7 as 


t co n 
wee “lel =. —1) z cos( 8) e- V TFI? || 
azi os T? |p? p 


In this case, our time variable t was their spatial variable x — £. 


12. Find the inverse of the Fourier transform 


Pe = [of an 


where L, 6, and y are real and positive and sgn(z) = 1 if R(z) > 0 and —1 if R(z) < 0 


Step 1: From the definition of the Fourier transform, show that 


1 co itw 1 itz 
fO= j: : dw = $ f dz. 
on wos wL 2T C cos{ zL 


cos{ BliFiysgn(w)] } Bll +iysen(2)] } 


Step 2: Show that the integral has simple poles at z,4 = +(2n — 1)8r/(2L) + (2n — 
1)i6yr/(2L), where n = 1,2,3,... 
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Step 3: For t > 0, use the residue theorem and show that 


co ettz ettz 
f(t) =i 5 Res ; Zn+ | + Res eae I 
n=1 cosd aro) cosd arien) 
Bll+iysgn(2)] Bll+iysgn(2)] 
where 
itz n 
Res( e int] _ es [1 + iysgn(zn)]e- C” -DEY L)+(2n-1)Prit/ (21), 
cos{ ste} L 
Bll +iysgn(z)] 


Step 4: For t < 0, show that f(t) =0. Why? 
Step 5: Show that we can summarize the results from Step 3 and Step 4 by 


eee 


z H) XO (HL)? tte Cr “VEIL fy cos[(2n — 1)8rt/2L] + sin[(2n — 1)8rt/2L]} . 


n= 


f(t) 


pa 


Use the residue theorem to verify the following integrals: 


© sin(a) T. © cos(x) T 
13. dx = 2 14. dz = 
: i ee +4c+5— gon ) J (x? + 1)? T= Be 
x sin(ax) - ® x? cos(ax) T Sh 
15 d y 16. = 1 bje" 
i xr? +4 A . i (x? + b?)? TA je 


where a,b > 0. 


17. The concept of forced convection is normally associated with heat streaming through 
a duct or past an obstacle. Bentwich showed that a similar transport can exist when 
convection results from a wave traveling through an essentially stagnant fluid. In the process 
of computing the amount of heating, he proved the following identity: 


°° cosh(hz)— 1 z 
S. ERZ colar) de = Infooth(]alm/A)], A> 0. 


Confirm his result. 


°° cosh(ha) — 1 cosh(hz)— 1 ai 
AA ON E a dr = tain E E AR 
i x sinh(ha) tostae) da n(f z sinh(hz) a 


Step 1: 


if a > 0 and C is a semicircle of infinite radius in the upper half-plane. 


Step 2: Within the contour, show that there is a removal singularity at z = 0 and simple 
poles at hz, = nmi with n = 1,2,3,.... 


5 Bentwich, M., 1966: Convection enforced by surface and tidal waves. Int. J. Heat Mass Transfer, 9, 
663-670. 
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Step 3: Show that 


cosh(hz)— 1 ai paa cosh(hz)— 1 «iz nti 
aiz dz = 92 R aiz. 
f, z sinh(hz) e í RÈ e| z sinh(hz) C h 


with 


[cosh(hz)—-1 aiz nmi] _ 1- (1) emai 
Res| z sinh(hz) e h ~ ani 


Step 4: Show that 


x sinh(ha) 2m—1 


~ cosh(hx) — 1 > —(2m — 1)ra/h 

J coshta) = cos(ax) dx = 4 5 SURA Ta) = ln[coth(ra/h)]. 

Z709 m=1 

Step 5: Redo the analysis if we replace a by —a. Reconcile your results with those given 
by Bentwich. 


2.2 INVERSION OF LAPLACE TRANSFORMS BY CONTOUR INTEGRATION 


Partial fractions and convolution are two common methods for finding the inverse of 
the Laplace transform F(s). In many instances these methods fail simply because of the 
complexity of the transform to be inverted. In this section we shall show how we can invert 
transforms through the powerful method of contour integration. Of course, the student 
must be proficient in the use of complex variables. 

Consider the piece-wise differentiable function f(x), which vanishes for x < 0. We can 
express the function e~°* f(a) by the complex Fourier representation of 


Herad as [fo etree at dw, (2.2.1) 


27 —oo 0 


for any value of the real constant c, where the integral 


r= f eT" F(t)| dt (2.2.2) 


exists. By multiplying both sides of Equation 2.2.1 by e° and bringing it inside the first 
integral, 


f(a) K le elctwi)x f fte (cree a| dw. (2.2.3) 


~ On = 
With the substitution z = c+ wi, where z is a new, complex variable of integration, 


tes = - e** pi f®He~ a| dz. (2.2.4) 


PA E ENE 


The quantity inside the square brackets is the Laplace transform F(z). Therefore, we can 
express f(t) in terms of its transform by the complex contour integral 


ra- [T rora (2.2.5) 


271 Jei 
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An outstanding mathematician at Cambridge University at the turn of the twentieth century, 
Thomas John I’Anson Bromwich (1875-1929) came to Heaviside’s operational calculus through 
his interest in divergent series. Beginning a correspondence with Heaviside, Bromwich was able to 
justify operational calculus through the use of contour integrals by 1915. After his premature death, 
individuals such as J. R. Carson and Sir H. Jeffreys brought Laplace transforms to the increasing 
attention of scientists and engineers. (Portrait courtesy of the Royal Society of London.) 


This line integral, the Bromwich integral,® runs along the line x = c parallel to the imaginary 
axis and c units to the right of it, the so-called Bromwich contour. We select the value of c 
sufficiently large so that the integral, Equation 2.2.2, exists; subsequent analysis shows that 
this occurs when c is larger than the real part of any of the singularities of F(z). 

We must now evaluate the contour integral. Because of the power of the residue 
theorem in complex variables, the contour integral is usually transformed into a closed 
contour through the use of Jordan’s lemma. See Section 2.1, Equations 2.1.4 and Equation 
2.1.5. The following examples will illustrate the proper use of Equation 2.2.5. 


e Example 2.2.1 


Let us invert 
e738 


F(s)= Psi) 


(2.2.6) 


6 Bromwich, T. J. PA., 1916: Normal coordinates in dynamical systems. Proc. London Math. Soc., 
Ser. 2, 15, 401-448. 
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Figure 2.2.1: Contours used in the inversion of Equation 2.2.6. 


From Bromwich’s integral, 


1 c+oot e(t-3)z 1 e(t-3)2 1 e(t-3)z 
fo = F z?(z — 1) u= 2ri f, z2?(z — 1) ag 2ri A z?(z — 1) Ay. (R2) 


—oot 


where CR is a semicircle of infinite radius in either the right or left half of the z-plane and 
C is the closed contour that includes Cr and Bromwich’s contour. See Figure 2.2.1. 

Our first task is to choose an appropriate contour so that the integral along Cr vanishes. 
By Jordan’s lemma, this requires a semicircle in the right half-plane if t— 3 < 0 and 
a semicircle in the left half-plane if t — 3 > 0. Consequently, by considering these two 
separate cases, we force the second integral in Equation 2.2.7 to zero and the inversion 
simply equals the closed contour. 

Consider the case t < 3 first. Because Bromwich’s contour lies to the right of any 
singularities, there are no singularities within the closed contour and f(t) = 0. 

Consider now the case t > 3. Within the closed contour in the left half-plane, there is 
a second-order pole at z = 0 and a simple pole at z = 1. Therefore, 


K e(t-3)2 i R e(t—3)2 i 
t) = : t ; wA 
0 = Res aa] +e Sey] or 
where 
e(t—3)z d e(t-3)z (t = 3)Jet-3)7 e(t—3)2 
nie EE ord aij OB Wed es] nag = 
Re sey Pb dz f e 1 | z-1 = ee 
(2.2.9) 
and 
R e(t-3)2 i i i e(t—3)z a 
> 1| = li — 1). = e’”. 2.2.10 
es| 1)’ zl (z Pan 1) ( ) 


Taking our earlier results into account, the inverse equals 


f(t) = [e7 — (t — 3) — 1] H(t — 3), (2.2.11) 
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(c,0) 


Figure 2.2.2: Contours used in the inversion of Equation 2.2.12. 


which we would have obtained from the second shifting theorem and tables. 


e Example 2.2.2 


For our second example of the inversion of Laplace transforms by complex integration, 


let us find the inverse of i 


ssinh(as)’ 


F(s) = (2.2.12) 


where a is real. From Bromwich’s integral, 


roz [xe (2.2.13) 


~ Qi Jei -Z Sinh(az) 


Here c is greater than the real part of any of the singularities in Equation 2.2.12. Using the 


infinite product for the hyperbolic sine,” 
tz tz 
2 = ú ; (2.2.14) 
zsinh(az) az?[1 + a?z?/r?][1 + a?z?/(4r?)][1 + a2z2/ (977) --- 
Thus, we have a second-order pole at z = 0 and simple poles at zn = -+enmi/a, where 


m= 1,2,3,.... 

We can convert the line integral Equation 2.2.13, with the Bromwich contour lying 
parallel and slightly to the right of the imaginary axis, into a closed contour using Jordan’s 
lemma through the addition of an infinite semicircle joining 100 to —ioo, as shown in Figure 
2.2.2. We now apply the residue theorem. For the second-order pole at z = 0, 


T Gradshteyn, I. S., and I. M. Ryzhik, 1965: Table of Integrals, Series and Products. Academic Press, 
Section 1.431, Formula 2. 
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el” 1 d [| (z—0)?e d ze” 
;0| = — li = lim — |- 2.2.1 
Be |- sinh(az)’ ol T250 dz | z sinh(az) 240 dz | ( 5) 
tz t tz sh tz 
Siak TE az cos or at (2.2.16) 
z>0 |sinh(az) sinh(az) sinh“ (az) a 
after using sinh(az) = az + O(z°). For the simple poles zn = +nri/a, 
et z— Zn)e* et 
Res | ———— ; z,,} = lim ——— = li 2.2.1 
or |- sinh(az)’ 7 stan Z sinh(az) item sinh(az) + az cosh(az) ( 7) 
exp(+nmit/a) 


= cane (2.2.18) 


because cosh(+n7i) = cos(nz) = (—1)”. Thus, summing up all of the residues gives 


f(t) = t 5 (—1)” exp(nrit/a) Ss (—1)” expl-nrit/a) 


SE i (2.2.19) 
a nri nri 
n=1 n=1 
t 2261), 
=- + t/a). 2.2.20 
re > a sin(nrt/a) ( ) 


In addition to computing the inverse of Laplace transforms, Bromwich’s integral places 
certain restrictions on F(s) in order that an inverse exists. If a denotes the minimum value 
that c may possess, the restrictions are threefold. First, F(z) must be analytic in the 
half-plane z > a, where z = x + iy. Second, in the same half-plane it must behave as z~*, 
where k > 1. Finally, F(x) must be real when z > a. 


e Example 2.2.3 


Is the function sin(s)/(s? + 4) a proper Laplace transform? Although the function 
satisfies the first and third criteria listed in the previous paragraph on the half-plane x > 2, 
the function becomes unbounded as y — +oo for any fixed x > 2. Thus, sin(s)/(s? + 4) 
cannot be a Laplace transform. 


e Example 2.2.4 


An additional benefit of understanding inversion by the residue method is the ability 
to qualitatively anticipate the inverse by knowing the location of the poles of F(s). This 
intuition is important because many engineering analyses discuss stability and performance 
entirely in terms of the properties of the system’s Laplace transform. In Figure 2.2.3 we 
have graphed the location of the poles of F(s) and the corresponding f(t). The student 
should go through the mental exercise of connecting the two pictures. 


8 For the proof, see Churchill, R. V., 1972: Operational Mathematics. McGraw-Hill, Section 67. 
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| s-plane ft) 
t 
s-plane ft) 
x = 
t 
| s-plane ft) 
t 
| s-plane ft) 
t 
s-plane fit) 
x 
t 
x 
s-plane fA) 
x 
t 
x 


Figure 2.2.3: The correspondence between the location of the simple poles of the Laplace transform F(s) 


and the behavior of f(t). 


98 Advanced Engineering Mathematics: A Second Course 


Problems 
Use Bromwich’s integral to invert the following Laplace transforms F(s): 


s+1 1 1 4 1 e* 


E (s+2)}(s+3) ~~ (s+a) ` s(s-— 2)’ ` s(s +a)? (s2? + 6?) i s? (s +2) 


6. Use Bromwich’s integral to invert 


1 
MO) = Fee) 


Step 1: Show that the singularities are all simple poles and are located at z = 0 and 
zn = +(2n — 1)ri/a, where n = 1,2,3,.... 


Step 2: Show that the corresponding residues are 


K 1 tz: + 2 2 1 t 
Res € -O| =~, and Res Lr G O t PA eet exp[+(2n )zi /a] 
z(1 + e727) 2 z(1 + e727) Onam 


Step 3: Show that the inverse Laplace transform equals 


5 sin[(2n — 1)rt/a] 
(Qn-1)r ` 
ee 
7. Use Bromwich’s integral to invert 
1 
F(s) = ——_—_.. 
(8) (s + b) cosh(as) 
Step 1: Show that the singularities are all simple poles and are located at z = —b and 
Zn = £(2n — 1)ri/ (2a), where n = 1,2,3,... because cosh(az) = cos(iaz) = 0. 


Step 2: Show that the corresponding residues are 


tz —bt 
Res| = : | 2 


(z + b) cosh(az)’ ~ cosh(ab)’ 


and 
tz 


e ts | = exp[+(2n — 1)mit/(2a)] . 
(z +b) cosh(az) ” ~ ailb (2n — 1)ni/(2a)| sin[(2n — 1)7r/2] 


Res | 


Step 3: Show that the inverse Laplace transform equals 


f = 


gab Da 7 sin[(2n — 1)rt/(2a)] 


aie ab) 4a?b? + (2n — 1)?r? 


n (2n — 1)r cos|(2n — ude) 
oe 4a?b? + (2n — 1)?r i 
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8. Use Bromwich’s integral to invert 


PUS ETEDI 


Step 1: Show that the singularities are all simple poles and are located at z = 0 and 
zn = +2nri/a, where n = 1,2,3,.... 


Step 2: Show that the corresponding residues are 


tz 


t 1 tz +49 4 
Res 0 = -+-+ , and Res Ta zia exp( dak /a) 
z(1 — e797) a 2 z(1 — e-%) nri 


Hint: Near z = 0, show that 


t 
a e e (ee a) 


z(1 — e797) = az*(1—az/2+--:) ~ az 2 


and read off the residue from the Laurent expansion. 


Step 3: Show that the inverse Laplace transform equals 


f(t) = ! m : | > La 
n=1 


9. Consider a function f(t) that has the Laplace transform F(z), which is analytic in the 
half-plane R(z) > sọ. Can we use this knowledge to find g(t), whose Laplace transform 
G(z) equals F'[y(z)], where y(z) is also analytic for R(z) > so? The answer to this question 
leads to the Schouten®?—Van der Pol!® theorem. 


Step 1: Show that the following relationships hold true: 


lore) ctoot 
G(z) = Fly(z)] = f fir)? dr, and = g(t) = = / Fly(z)\e® dz. 


Tt —ocot 


Step 2: Using the results from Step 1, show that 


love) 1 ctoot 
g(t) = f F(T) Fal e7?) etz a| dr. 
0 c—oot 


This is the Schouten-Van der Pol theorem. 
Step 3: If G(z) = F(,/z) show that 


= wail ritrex(—2) ap 


9 Schouten, J. P., 1935: A new theorem in operational calculus together with an application of it. 
Physica, 2, 75-80. 


10 Van der Pol, B., 1934: A theorem on electrical networks with applications to filters. Physica, 1, 
521-530. 
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Hint: Do not evaluate the contour integral. Instead, ask yourself: What function of time 
has a Laplace transform that equals e~°)7, where r is a parameter? Then use tables. 


2.3 INTEGRAL EQUATIONS 
An integral equation contains the dependent variable under an integral sign. The convo- 


lution theorem provides an excellent tool for solving a very special class of these equations, 
the Volterra equation of the second kind:'+ 


f(t) - f K{t, x, f(x)| dx = g(t), 0<t<T. (2.3.1) 


These equations appear in history-dependent problems, such as epidemics,!? vibration prob- 
lems,!° and viscoelasticity. 4 


e Example 2.3.1 


Let us find f(t) from the integral equation 


f(t) =4t- 3 | f(a) sin(t — x) dz. (2.3.2) 


The integral in Equation 2.3.2 is such that we can use the convolution theorem to find 
its Laplace transform. Then, because L[sin(t)| = 1/(s?+1), the convolution theorem yields 


gl f oea da! = (2.3.3) 


Therefore, the Laplace transform converts Equation 2.3.2 into 


4 3F(s) 


F(s === : 2.3.4 
(s) s? s2+1 ( ) 
Solving for F(s), 
A(s? +1) 
F(s) = —. 2.3.5 
(s) 52(s2 +4) (25.5) 
By partial fractions, or by inspection, 
1 3 
F(s) = — + ——. 2.3.6 
(s) g” g s2 +4 ( ) 


1 Fock, V., 1924: Uber eine Klasse von Integralgleichungen. Math. Z., 21, 161-173; Koizumi, S., 1931: 


On Heaviside’s operational solution of a Volterra’s integral equation when its nucleus is a function of (x — £). 
Philos. Mag., Ser. 7, 11, 432-441. 


2 Wang, F. J. S., 1978: Asymptotic behavior of some deterministic epidemic models. SIAM J. Math. 
Anal., 9, 529-534. 


3 Lin, S. P., 1975: Damped vibration of a string. J. Fluid Mech., 72, 787-797. 


4 Rogers, T. G., and E. H. Lee, 1964: The cylinder problem in viscoelastic stress analysis. Q. Appl. 
Math., 22, 117-131. 


Advanced Transform Methods 
Therefore, inverting term by term, 
f(t) = t + 3 sin(2¢). 
Note that the integral equation 
t 
f(t) = 4t — 3 | f(t — x) sin(a) dx 

0 

also has the same solution. 


e Example 2.3.2 


Let us solve the equation 


f(t)+ a? f f(r) dr = B — C cos(wt), f(0) =0. 
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(2.3.7) 


(2.3.8) 


(2.3.9) 


Again the integral is one of the convolution type; it differs from the previous example 


in that it includes a derivative. Taking the Laplace transform of Equation 2.3.9, 


a?F(s) B sC 


s s sw? 


Because f(0) = 0, Equation 2.3.10 simplifies to 


Cs? 
2 2 
Solving for F(s), 

B Cs? 


PUIS s2 +a? (s? +a?)(s? +w?) 


Using partial fractions to invert Equation 2.3.12, 


ac C 
f(t) = (= Æ =) sin(at) — = sin(wt). 


e Example 2.3.3 


Let us solve! the integral equation 


f(t) = a ah f(it—2x)f(z)de+e™*. 


(2.3.10) 


(2.3.11) 


(2.3.12) 


(2.3.13) 


(2.3.14) 


15 Hounslow, M. J., 1990: A discretized population balance for continuous systems at steady state. 


AICHE J., 36, 106-116. 


102 Advanced Engineering Mathematics: A Second Course 


Taking the Laplace transform of Equation 2.3.14, we obtain 


F(s) = en Mie ` 7 (2.3.15) 


Solving for F(s) so that F(s) + 0 as s > ov, we have 


F(s) = 20+! CEECEE 2a+1 Ca alae 
sS) = = . 
a 


a a (Qa+1)(s+1) a s+1 
(2.3.16) 
Taking the inverse of Equation 2.3.16, 
2a +1 vV2a+1 
f(t) = d(t) g(t), (2.3.17) 
a a 
where g(t) is the inverse of the Laplace transform G(s), 
2 1 1 1/(1 +2 
gigs VEO a aa (2.3.18) 
s+1 Vs+1Iys+1/(2a+1) 
E raa Eea (2.3.19) 
vV2a+1 = 
and i 
A(s) = ; 2.3.20 
(s) Vs+1/s+1/(2a +1) ( ) 
Taking the inverse of H (s), we find that 
a+1 at 
h(t) = ——t } Io| —— 2.3.21 
2 ep( i) (soc) ( ) 
and 
+1 a+1 at a a+1 at 
h(t) =-= t) I t) 
(t) el 2a+1 ) (sect) = zal 2a 41 ) (a) i 
2.3.22) 


where Io(-) and Jı (-) are modified Bessel functions of the first kind. 
Because sH (s) = L{h’(t)]+h(0) and h(0) = 1, h'(t) = L7HsH (s)| — ô(t) or L~1[sH(s)] 
= h' (t) + ô(t). Then, 


t aa 
g(t) = vV2a i [wns oO Fae (2.3.23) 
a a+1 at 
= 2a +1 fat | se ott) n(s44) 
a a+1 at 
zal ttu) a(z) C 


Finally, substituting Equation 2.3.24 into Equation 2.3.17, 


f(t) = ep -4 44) ee z) ns :)| l (2.3.25) 
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Problems 
Solve the following integral equations: 
1. = 2 i — x)e™?” dx 2: =1 xz) sin(t — x) dx 
fl =1+2f t-a) f= 1+ | fesat- a) 
= — zje "dz 4. = At? — —az)e"d 
ats | St-a) fl =4P— | faye de 
ose +f f(x) sin(t — x) dx 6. f(t) = 8t? — 3 | f(x) sin(t — x) dx 


7. fŒ) =- 2f f(t — x) sinh(2z) dx 8. f(t) =1+ i: f(t — x) cos(x) dx 
9. f(t) = e% + a, f(t — x) cos(x) dx 10. f(t) =t? + [ f(a) sin(t — x) dx 
11. f(t) ETEC ) cos(t — a) dz 12. f(t) + f rex \(t— x)dzr =t 
13. f(t) m f(x)(t— 2)? dx 14. f(t) =avi- f fe = 


15. Solve the following equation for f(t) with the condition that f(0) = 
t 
F =t 1 f(t — x) cos(x) dz. 
0 
16. Solve the following equation for f(t) with the condition that f(0) = 


f' (t) = sin(t J+ f FE a) cosl ) cos(x) dx 


17. During a study of nucleation involving idealized active sites along a boiling surface, 
Marto and Rohsenow’® solved the integral equation 


to find the position x(t) of the liquid/vapor interface. If A, B, and C are constants and 
x(0) = 0, find the solution for them. 


18. Solve the following equation for x(t) with the condition that x(0) = 0: 


a(t) +t= ce : Vie 
where c is constant. 
Step 1: Show that 
X(s) = c = e(c+ys) 


s2(c= ys) se-s) 


16 Marto, P. J., and W. M. Rohsenow, 1966: Nucleate boiling instability of alkali metals. J. Heat 
Transfer, 88, 183-193. 
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Step 2: Use partial fractions to show that 


X(s) = 5 (1+ £) (5 +2 =) 


Step 3: Show that 
E [1 + erf (evé)] - et- 1- 2ey/ Ek, 
oa T 


19. During a study of the temperature f(t) of a heat reservoir attached to a semi-infinite 
heat-conducting rod, Huber!” solved the integral equation 


e PTO, 
VT 0 Vt eam! Ob 
where a and 8 are constants and f(0) = 0. Find f(t) for him. 
Step 1: Show that 


f(t) =a 


T, 


= x = = ae 
F(s) = 53/2 (51/24 8) s(s— 8?) 83/2 (s — B2) 


Q 
~ PEP Ps Sp 


Step 2: Taking the inverse term by term, show that 


a 2 Ac8"t nye 2 
fj) = ea e” x? dx 
p? VT Jo 


g t pt PAGE , 
=e eft 1 4 28i w | e” dx]. 
Vu Va 0 


20. During the solution of a diffusion problem, Zhdanov, Chikhachev, and Yavlinskii!® 
solved an integral equation similar to 


[1o [1 — erf (avt -7 )]| dr = at, 


where erf(x) = = J ew dy is the error function. What should they have found? 
T Jo 


Step 1: Show that 


a a? a? af 


+ 
svs +a? s2ys+a? 


17 Huber, A., 1934: Eine Methode zur Bestimmung der Wärme- und Temperaturleitfähigkeit. Monatsh. 
Math. Phys., 41, 35-42. 


18 Zhdanov, S. K., A. S. Chikhachev, and Yu. N. Yavlinskii, 1976: Diffusion boundary-value problem for 
regions with moving boundaries and conservation of particles. Sov. Phys. Tech. Phys., 21, 883-884. 
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Step 2: Show that 


Lit erf(avt) — atla vt) + vi g” 


ayn. 


d a 1 1 a 
© ds ap 2asyYs+a2?  2a(s+a?)3/2 ~~ 52./s +a? 


Step 3: Taking the inverse of Step 1 term by term, show that 


3 7 

t 
f(t) =a+a%t+ Saerf(Vat) + at erf(Vat) + = goer, 

T 

21. The Laguerre polynomial!’ 
e d” 
t) = Lalt) = =- (t e™ =Q 2: 
u(t) = Inl) = FE (e), n=0,1,2,3, 


satisfies the ordinary differential equation 
ty” + (1-t)y' + ny = (ty’)! — ty’ + ny =0, 


with y(0) = 1 and y’(0) = —n. 


Step 1: Using the properties that L[f((t)] = s"F(s) — s"—!f(0) —--- — sf@-2)(0) — 
f@-(0) and Equation L[t f(t)] = —F’(s), show that the Laplace transformed version of 
this differential equation is 
j n+1l-s n n+l 
Y (s) = oa Y (s) = Y (s5) Y (5), 
s(s—1) s—1 s 


where Y (s) is the Laplace transform of y(t). 


Step 2: Using the property that L[t f(t)] = —F’(s) and the convolution theorem, show that 
Laguerre polynomials are the solution to the integral equation 


ty(t) = (n+ Df y(T)dr — net f y(t)e* dr. 


2.4 THE SOLUTION OF THE WAVE EQUATION BY USING LAPLACE TRANSFORMS 


The solution of linear partial differential equations by Laplace transforms is the most 
commonly employed analytic technique after separation of variables. Because the transform 
consists solely of an integration with respect to time, the transform U (x, s) of the solution 
of the wave equation u(x,t) is 


U(a,s) = [ u(a, the” ™ dt, (2.4.1) 


19 See Section 5.3 in Andrews, L. C., 1985: Special Functions for Engineers and Applied Mathematicians. 
MacMillan, 357 pp. 
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assuming that the wave equation only varies in a single spatial variable x and time t. 
Partial derivatives involving time have transforms similar to those that we encountered 
in the case of functions of a single variable. They include 


Liu: (a, t)] = sU (x, s) — u(a, 0), (2.4.2) 


and 
Llu (a, t)] = U (x, s) — su(x,0) — u(x, 0). (2.4.3) 


These transforms introduce the initial conditions via u(z,0) and u,(a,0). On the other 
hand, derivatives involving x become 


Clus(2,t)) = Flue, = TE, (2.4.4) 
or ae dU (a, s) 
Llure (2, t)] = qa Elele, t)] Te ar, (2.4.5) 


Because the transformation eliminates the time variable, only U (x, s) and its derivatives 
remain in the equation. Consequently, we transform the partial differential equation into a 
boundary-value problem involving an ordinary differential equation. Because this equation 
is often easier to solve than a partial differential equation, the use of Laplace transforms 
considerably simplifies the original problem. Of course, the Laplace transforms must exist 
for this technique to work. 

The following schematic summarizes the Laplace transform method: 


In the following examples, we illustrate transform methods by solving the classic equa- 
tion of telegraphy as it applies to a uniform transmission line. The line has a resistance R, 
an inductance L, a capacitance C', and a leakage conductance G per unit length. We denote 
the current in the direction of positive x by I; V is the voltage drop across the transmission 
line at the point x. The dependent variables I and V are functions of both distance x along 
the line and time t. 

To derive the differential equations that govern the current and voltage in the line, 
consider the points A at x and B at x + Az in Figure 2.4.1. The current and voltage at 
A are I(x,t) and V(z,t); at B, I + Sl Ax and V + X Ar. Therefore, the voltage drop 
from A to B is -X Axr and the current in the line is J + SI Ar. Neglecting terms that are 
proportional to (Az)?, 

ol OV 
G + Rt) Ag = -At (2.4.6) 
The voltage drop over the parallel portion HK of the line is V while the current in this 
portion of the line is -2l Ar. Thus, 


OV ol 
= Ag = ——Azg. 2.4. 
(c J + av) x JA (2.4.7) 
Therefore, the differential equations for J and V are 
ol OV 
L + RI = 2.4.8 
Ot á Ox’ ( ) 
g av ar 
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A H B 
a TOT -\N\V — 
I L Ax R Ax r+ 34 ay 
8x 


at =]. vV 
V Càr - G Ax V+ Sx Ax 


x K x+Ax 
Figure 2.4.1: Schematic of a uniform transmission line. 


Turning to the initial conditions, we solve these simultaneous partial differential equa- 
tions with the initial conditions 
I(x,0) = Io(x), (2.4.10) 


and 
V(a,0) = Vo(a) (2.4.11) 


for 0 < t. There are also boundary conditions at the ends of the line; we will introduce 
them for each specific problem. For example, if the line is short-circuited at z = a, V = 0 
at x = a; if there is an open circuit at x =a, I = 0 at x = a. 

To solve Equation 2.4.8 and Equation 2.4.9 by Laplace transforms, we take the Laplace 
transform of both sides of these equations, which yields 


dV (x, 8) 


(Ls + R)I (a, s) = — „z t LIo(x), (2.4.12) 
and = 
E T - ) OnE: (2.4.13) 


Eliminating T gives an ordinary differential equation in V 


dV I dIo(x) 
eae ae Ty 
daz 4 4 dx 


C(Ls + R)Vo(z), (2.4.14) 


where q? = (Ls + R)(Cs + G). After finding V, we may compute J from 


T= i 
Is+Rdx Ls+R 


(2.4.15) 


At this point we treat several classic cases. 
e Example 2.4.1: The semi-infinite transmission line 


We consider the problem of a semi-infinite line 0 < x with no initial current and charge. 
The end x = 0 has a constant voltage E for 0 < t. 
In this case, 
PV as 
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The boundary conditions at the ends of the line are 
VO,t)=£, 0<t, (2.4.17) 


and V (x,t) is finite as oo. The transform of these boundary conditions is 


V(0,s) = E/s, and lim V(a,s) > 0. (2.4.18) 


T00 


The general solution of Equation 2.4.16 is 
V(a,s) = Ae + Be. (2.4.19) 


The requirement that V remains finite as £ — oo forces B = 0. The boundary condition at 
x = 0 gives A = E/s. Thus, 


V(x,s) = = exp |-Vis +R)(Cs+G)a}. (2.4.20) 


We discuss the general case later. However, for the so-called “lossless” line, where R = G = 
0, 


E 

V(x,s) = — exp(—sz/c), (2.4.21) 
s 

where c = 1/VLC. Consequently, 


V(x,t) = EH(t = Z) , (2.4.22) 
c 

where H(t) is Heaviside’s step function. The physical interpretation of this solution is as 
follows: V (x,t) is zero up to the time z/c, at which time a wave traveling with speed c from 


x = 0 would arrive at the point x. V (x,t) has the constant value E afterwards. 
For the so-called “distortionless” line, R/L = G/C = p, 


V(z,t) = Belii = =) (2.4.23) 
c 
In this case, the disturbance not only propagates with velocity c but also attenuates as we 
move along the line. 

Suppose now, that instead of applying a constant voltage E at x = 0, we apply a 
time-dependent voltage, f(t). The only modification is that in place of Equation 2.4.20, 


V(x, s) = F(s)”. (2.4.24) 


In the case of the distortionless line, q = (s + p)/c, this becomes 


V(a, 8) = F(s)e7tP)e/e (2.4.25) 
and P m 
V(z,t) = E = =) H(t = Z) . (2.4.26) 


20 Prechtl and Schürhuber (Prechtl, A., and R. Schürhuber, 2000: Nonuniform distortionless transmission 
lines. Electr. Eng. [Berlin], 82, 127-134) generalized this problem to nonuniform transmission lines. 
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V(x,t) 


i. ee twice thrice reflected 
E i —— _—— “a 
i i i aes 
| (I-x)/c (l+x)/c (3l-x)/c (3l+x)/c 


Figure 2.4.2: The voltage within a lossless, finite transmission line of length l as a function of time t. 


Thus, our solution shows that the voltage at x is zero up to the time w/c. Afterwards V (x,t) 
follows the voltage at x = 0 with a time lag of x/c and decreases in magnitude by e~?*/°. 


e Example 2.4.2: The finite transmission line 


We now discuss the problem of a finite transmission line 0 < x < l with zero initial 
current and charge. We ground the end x = 0 and maintain the end x = l at constant 
voltage E for 0 < t. 

The transformed partial differential equation becomes 


V = 6 
aaa gv =0, O<2<l. (2.4.27) 
The boundary conditions are 
V(0,t)=0, and V(l, t) = E, 0x t: (2.4.28) 


The Laplace transform of these boundary conditions is 


V(0,s)=0, and V(l,s)= E/s. (2.4.29) 
The solution of Equation 2.4.27 that satisfies the boundary conditions is 


_ E sinh(qz) 


V(x, s) = Sahay (2.4.30) 


Let us rewrite Equation 2.4.30 in a form involving negative exponentials and expand the 
denominator by the binomial theorem, 


V(z,s) = ae Gane 
E 

_ Zea) Sere) (1 e724 | e 4a --+) (2.4.32) 
Ss 

x E [e~ae-®) = e alta) ae e7 g(3l—a) Pes e7 46l+z) + ‘| E (2.4.33) 
S 


In the special case of the lossless line where q = s/c, 


V(x, s) = E [ere aye = e 8 +a)/e + e7 8(3l—2)/c _ e 88l+2)/c Ie ‘|, (2.4.34) 


Veen) = B] H(t = ae f°) eae 1) ae ate) a, 
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Figure 2.4.3: The voltage within a submarine cable as a function of distance for various values of Kt. 


We illustrate Equation 2.4.35 in Figure 2.4.2. The voltage at x is zero up to the time 
(l — x)/c, at which time a wave traveling directly from the end x = l would reach the point 
x. The voltage then has the constant value Æ up to the time (l+x)/c, at which time a wave 
traveling from the end x = l and reflected back from the end x = 0 would arrive. From this 
time up to the time of arrival of a twice-reflected wave, it has the value zero, and so on. 


e Example 2.4.3: The semi-infinite transmission line reconsidered 


In the first example, we showed that the transform of the solution for the semi-infinite 
line is 


E 
V(z,s) = =e, (2.4.36) 


where q? = (Ls + R)(Cs+G). In the case of a lossless line (R = G = 0), we found traveling 
wave solutions. 
In this example, we shall examine the case of a submarine cable,?! where L = G = 0. 
In this special case, 
= E E 
V(x, s) =e v s/s, (2.4.37) 
s 


where x = 1/(RC). From a table of Laplace transforms,?? we can immediately invert 
Equation 2.4.37 and find that 


V(a,t)=E e(z) y (2.4.38) 


where erfc(-) is the complementary error function. Unlike the traveling wave solution, the 
voltage diffuses into the cable as time increases. We illustrate Equation 2.4.38 in Figure 
2.4.3. 


21 First solved by Thomson, W., 1855: On the theory of the electric telegraph. Proc. R. Soc. London, 
Ser. A, 7, 382-399. 


22 See Churchill, R. V., 1972: Operational Mathematics. McGraw-Hill Book, Section 27. 
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e Example 2.4.4: A short-circuited, finite transmission line 


Let us find the voltage of a lossless transmission line of length l that initially has the 
constant voltage E. At t = 0, we ground the line at x = 0 while we leave the end x = l 
insulated. 

The transformed partial differential equation now becomes 


CV 2 sE 
= 2.4.39 
dz? K ce’ ( ) 
where c = 1/ VLC. The boundary conditions are 
V(0,s) = 0, (2.4.40) 
and _ 
= 1 dV(l, s) 
I(l = =0 2.4.41 
ls) =- (2.4.41) 
from Equation 2.4.15. 
The solution to this boundary-value problem is 
ar E E h[s(l — 
V(z,5) = PESTA (2.4.42) 


s s cosh(sl/c) 


The first term on the right side of Equation 2.4.42 is easy to invert and the inversion equals 
E. The second term is much more difficult to handle. We will use Bromwich’s integral. 
In Section 2.2 we showed that 


_1fcosh[s(l—a)/e]| 1 etoi coshjz(l — x) /cle! 
L£ { s cosh(sl/c) \ E a z cosh(2l/c) dz. (2.4.43) 


— cot 


To evaluate this integral, we must first locate and then classify the singularities. Using the 
product formula for the hyperbolic cosine, 


42? (l-r)? 42? (l-r)? 
cosh[z(I-2)/e] _ [1+ FRH t Sher]. Gia 
z cosh(zl/c) z + R H 455)... i 


This shows that we have an infinite number of simple poles located at z = 0, and zp, = 
+(2n — 1)rci/(2l), where n = 1,2,3,.... Therefore, Bromwich’s contour can lie along, and 
just to the right of, the imaginary axis. By Jordan’s lemma we close the contour with a 
semicircle of infinite radius in the left half of the complex plane. Computing the residues, 


cosh[z(l — x) /c]et |, cosh[z(l— æ)/de®* _ 
me { z cosh(zl/c) o} — lim cosh(zl/c) =1, (2.4.45) 
and 
cosh[z(1 — x)/cjet* (z=) cosh[z(I — x) [cet 
> { z cosh(zl/c) sin g a z cosh(zl/c) (2.4.46) 


E cosh[(2n — 1r (l — x)i/(21)] exp[4(2n — 1)rcti/(21)] 
[(2n — 1)ri/2] sinh[(2n — 1)ri/2] 


(2.4.47) 
_ Bly (2n — 1)r(l — x)  (2n-— 1)reti 
~ nir 21 | on E 2l | 
(2.4.48) 
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Summing the residues and using the relationship that cos(t) = (e + e~")/2, 


V(z,t) =E gfi ‘ 3 (aa as cos| 2 —1)n(1— »] cos| Gn Meet} 


cael 2n—1 2l 2l 
(2.4.49) 
4E SS (-1)"*1 2n — 1)r(l — 2n — 1)rct 
= 5 G3 cos |£ Lae 2| cos Emar . (2.4.50) 
M 2n— 1 2l 2l 
An alternative to contour integration is to rewrite Equation 2.4.42 as 
me E e7 s2/c [1 ae ea 
V(a,s) = r fı (span (2.4.51) 
E = =ar = 
see [1 = sz/e ë s(2l—x)/c +e s(2l+2)/c see | (2.4.52) 
8 


so that 


e Example 2.4.5: The general solution of the equation of telegraphy 


In this example we solve the equation of telegraphy without any restrictions on R, C, 
G, or L. We begin by eliminating the dependent variable I(x, t) from the set of equations, 
Equation 2.4.8 and Equation 2.4.9. This yields 
0?V OV 8V 
L— L — + RGV = —. 2.4.54 
C PR + (GL + RC) a t a2 ( ) 
We next take the Laplace transform of Equation 2.4.54 assuming that V(x,0) = f(x), and 
V;(a,0) = g(a). The transformed version of Equation 2.4.54 is 


= 
a — (CLs? + (GL + RC)s + RG|V = —CLg(x) — (CLs +GL+RC)f(x), (2.4.55) 
L 
or ne j 5 í ) 
dV (s+p -0o g(a s 2p 
= 2.4.56 
dx? c2 V c2 (3 + c2 F(x), ( 5 ) 


where c? = 1/LC, p= e (RC + GL)/2, and o = c?(RC — GL)/2. 

We solve Equation 2.4.56 by Fourier transforms with the requirement that the solution 
dies away as |z| — oo. The most convenient way of expressing this solution is the convolution 
product 


V(x, s) = E $ ( y 2) ræ) els ENP T (2.4.57) 


c 24/ (s + p)? — 0? 


From a table of Laplace transforms, 


s2 — q2 


Wak a} ro hav? —0) H(t —b), (2.4.58) 
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where b > 0 and Io(-) is the zeroth-order modified Bessel function of the first kind. There- 
fore, by the first shifting theorem, 


exp[-lel Vs +p? — o/c] 
(s+ p)?—o? 


=e" Ig|o VE- J a] a(+- E). (2.4.59) 


Using Equation 2.4.59 to invert Equation 2.4.57, we have that 


V (x,t) = Le-P*g(x) * Io [o P- @70?] H(t —|z\/c) 


+ hefla) 2 { folo VP — lel} Hit lelo) 


+ ane «Iolo = (@/e?| H(t = lel/c) 
+ 5e [f(a + et) + f(e— et)]. (2.4.60) 


The last term in Equation 2.4.60 arises from noting that sF(s) = L[f(t)] + f(0). If we 
explicitly write out the convolution, the final form of the solution is 


V(a,t) = $e" [f(a + ct) + f(a — ct)] 


+ beef Wo + 20¢(a\lto[o VER = | an 


—ct 


+ kee f fo AAG være el} dn. (2.4.61) 


There is a straightforward physical interpretation of the first line of Equation 2.4.61. 
It represents damped progressive waves; one is propagating to the right and the other to 
the left. In addition to these progressive waves, there is a contribution from the integrals, 
even after the waves pass. These integrals include all of the points where f(x) and g(x) 
are nonzero within a distance ct from the point in question. This effect persists through all 
time, although dying away, and constitutes a residue or tail. Figure 2.4.4 illustrates this for 
p = 0.1, o = 0.2, and c = 1. This figure was obtained using the MATLAB script: 


% initialize parameters in calculation 
clear; dx = 0.1; dt = 0.5; rho_over_c = 0.1; sigma_over_c = 0.2; 
X=[-10:dx:10]; T = [O0:dt:10]; % compute locations of x and t 
for j=1:length(T); t = T(j); 
for i=1:length(X); x = X(i); 
XX(i,j) = x; TT(i,j) = t; detai = 0.05 % set up grid 
% compute characteristics x+ct and x-ct 
characteristic_l = x - t; characteristic_2 = x + t; 
% compute first term in Equation 2.4.61 
F = inline(’stepfun(x,-1.0001)-stepfun(x,1.0001)’); 
u(i,j ) = F(characteristic_1) + F(characteristic_2) ; 
% find the upper and lower limits of the integration 


upper = characteristic_2; lower = characteristic_1; 
if t > 0 & upper > -1 & lower < 1 

if upper > 1 upper = 1; end 

if lower < -1 lower = -1; end 


114 Advanced Engineering Mathematics: A Second Course 


5 
0 


€ 
-5 NY 


10 


Figure 2.4.4: The evolution of the voltage with time given by the general equation of telegraphy for initial 
conditions and parameters stated in the text. 


% 


% 
% 


set up parameters needed for integration 
interval = upper-lower; 
NN = interval / detai; 
if mod(NN,2) > O NN = NN + 1; end; 
deta = interval / NN; 
compute integrals in Equation 2.4.61 by Simpson’s rule 
sumi deals with the first integral while sum2 is the second 
sumi = 0; sum2 = 0; eta = lower; 
for k = 0:2:NN-2 

arg = sigma_over_c * sqrt (t*t-(x-eta)*(x-eta)); 

sumi = sumi + besseli(0,arg) ; 

if (arg == 0) 

sum2 = sum2 + 0.5 * sigma_over_c * t; 
else 


sum2 = sum2 + t * besseli(1,arg) / arg; end 
eta = eta + deta; 
arg = sigma_over_c * sqrt(t*t-(x-eta)*(x-eta)); 
sumi = sumi + 4*besseli(0,arg) ; 
if (arg == 0) 

sum2 = sum2 + 4 * 0.5 * sigma over- c * t; 
else 

sum2 = sum2 + 4 * t * besseli(1,arg) / arg; end 
eta = eta + deta; 
arg = sigma_over_c * sqrt(t*t-(x-eta)*(x-eta)); 
sumi = sumi + besseli(0,arg) ; 
if (arg == 0) 

sum2 = sum2 + 0.5 * sigma-over_c * t; 
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else 
sum2 = sum2 + t * besseli(1,arg) / arg; end 
end 
u(i,j) = u(i,j) + 2 * rho_over_c * deta * sum / 3... 
+ sigma_over_c * deta * sum2 / 3; 
end 
% multiply final answer by damping coefficient 
u(i,j) = 0.5 * exp(-rho_over_c * t) * u(i,j); 
end;end; 
% plot results 
mesh(XX,TT,real(u)); colormap spring; 
xlabel(’DISTANCE’ ,’Fontsize’,20); ylabel(’TIME’, ’Fontsize’ ,20) 
zlabel(’SOLUTION’ , ’Fontsize’ ,20) 


We evaluated the integrals by Simpson’s rule for the initial conditions f(x) = H(x +1) — 
H(x — 1), and g(x) = 0. If there was no loss, then two pulses would propagate to the left 
and right. However, with resistance and leakage the waves leave a residue after their leading 
edge has passed. 


Problems 


1. Use transform methods to solve the wave equation 


u Pu 
tae 1 t 
aR 522’ O0<a<l, 0<t, 
with the boundary conditions u(0,t) = u(1,t) = 0, 0 < t, and the initial conditions u(x, 0) = 


0, w(#,0) =1,0<a<1. 


Step 1: Take the Laplace transform of the partial differential equation and boundary con- 
ditions and show that 


d?U(z, 8) 
dx? 


subject to the boundary conditions U(0,s) = U(1,s) = 1. 


— s?U(a,s) = —1, 0<a<1, 


Step 2: Show that the solution to the previous step is 


_ 1—cosh(sz) | [cosh(s) — 1]sinh(sx) 
UG?) = i s? sinh(s) 


S 


Step 3: Show that U(x,s) has simple poles at sn = +nri with n = 1,2,3,... and a 
removable pole at s = 0. 


Step 4: Use Bromwich’s integral and show that 


u(z,t) = 4 5 sin[(2m — ae — Dat] 


m=1 


2. Use transform methods to solve the wave equation 


u u 


De Jr?’ 0O<z<l, 0<t, 
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with the boundary conditions u(0,t) = u,(1,t) = 0, 0 < t, and the initial conditions 
u(x, 0) = 0, uz(x,0) =2,0<a4 <1. 


Step 1: Take the Laplace transform of the partial differential equation and boundary con- 
ditions and show that 


dU(z, s) 
dx? 


with the boundary condition U(0,s) = U’(1,s) = 0. 


— s*U(z,s) = —7, 0<a<1, 


Step 2: Show that the solution to the previous step is 


xs cosh(s) — sinh(sz) 
s? cosh(s) 


U(x,s) = 


Step 3: Show that U(x, s) has simple poles at sn = +(2n — 1)ri/2 with n = 1,2,3,... and 
a removable pole at s = 0. 


Step 4: Use Bromwich’s integral and show that 


z 2 ae sin] y=] safe - wn l 


3. Use transform methods to solve the wave equation 


u Ou 

— = — 1 t 

E J2 O0<a<l, 0<t, 
with the boundary conditions u(0,t) = u(1,t) = 0, 0 < t, and the initial conditions u(x, 0) = 
sin(nx), us(x,0) = — sin(nrz), 0< x < 1. 


Step 1: Take the Laplace transform of the partial differential equation and boundary con- 
ditions and show that 


U(x, s8) 
dx? 


with the boundary conditions U (0, s) = U (1, s) = 0. 
Step 2: Show that the solution to the previous step is U(x, s) = (s — 1) sin(nx) /(s? + 7°). 


— 3U (z, s) = —s sin(nx) + sin(nx), 0<a<l, 


Step 3: Inverting by inspection, show that u(x,t) = sin(nx) cos(rt) — sin(7x) sin(at) /7. 


4. Use transform methods to solve the wave equation 
u 0u 
=, 
ot? Ox? 


with the boundary conditions u(0, t) = sin(wt), u(a, t) = 0, 0 < t, and the initial conditions 
u(x, 0) = u(x, 0) =0,0< x <a. Assume that wa/c is not an integer multiple of 7. Why? 


O0<a<a, 0<t, 


Step 1: Take the Laplace transform of the partial differential equation and boundary con- 
ditions and show that 
d?U(a,s) 3 


T2 — (2,8) =0, 0<a<a, 
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with the boundary condition U(0, s) = w/(s? +w?) and U(a,s) = 0. 


Step 2: Show that the solution to the previous step is 


w sinh[s(a — x) /c] 


U8) = (s2 +w?) sinh(sa/c)’ 
Step 3: Show that U(z,s) has simple poles at s = +wi and sn = +nrci/a with n = 
1,2,3,... and a removable pole at s = 0. 


Step 4: Use Bromwich’s integral and show that 


_ sin[w(a — x)/¢] nlu 2wa ©  sin(nrz/a) sin nret 
a sin(wa/c) sing) c Dp j ( a ): 


5. Use transform methods to solve the wave equation 


u Pu 


a E pr?’ 0<a<L, 0<t, 


with the boundary conditions u,(0,t) = — f (t), us(L, t) = 0, 0 < t, and the initial conditions 
u(x, 0) = ur(z,0) =0,0<a< L. 


Step 1: Take the Laplace transform of the partial differential equation and boundary con- 
ditions and show that 


aU 2 
aC.) _ Fu (@,s) =0, 0<a<L, 


with the boundary conditions U’(0,s) = —F (s) and U’(L, s) = 0. 


Step 2: Show that the solution to the previous step is 


cF(s) cosh[s(L — x)/c] 
s sinh(sL/c) 


U(a,s) = 


Step 8: Replacing sinh and cosh by their definitions and expanding the denominator as a 
geometric series, show that 


eee pete (1 4 e 28h/e 4 e—4sL/e 1 3 a) : 
Step 4: Multiplying everything out and inverting term by term, show that 


u(x,t) = DP f(t —x/c— 2nL/c) H(t — «/c— 2nL/c) 


n=0 


+c S f(t+a/ce-—2mL/c)H(t + z/c — 2mL/c). 


m=1 
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6. Use transform methods to solve the wave equation 


u 8u 


za =o ga lO a<a<b, 0<t, 


with the boundary conditions u(a,t) = us(b,t) = 0, 0 < t, and the initial conditions 
u(x, 0) = 0, ue(x,0) = —q(0), a < x < b. 
Step 1: Take the Laplace transform of the partial differential equation and boundary con- 
ditions and show that 
d?U(z,s) 
2 ’ 2 

c— ya ~ 8 Ulz,s) = sQ(s), a<a<b, 
with the boundary conditions U (a, s) = U’ (b, s) = 0. 
Step 2: Show that the eigenfunctions sin[kn (x — a)], where kn = (2n + 1)7/[2(b — a)] and 
n =0,1,2,..., satisfy the boundary conditions. 


Step 3: Expand the right side of the differential equation using an eigenfunction expansion 
consisting of sin[k,,(a — a)]. Show that 


_ 48Q(s) = | [2n +1)r(z-— a) 
g T dal sn| 2(b— a) | 


Step 4: Assuming that 


U(a,s) = X An sin 


n=0 


= s+ 2] 


show by direct substitution that 


r E 4sc?Q(s) [e (2n + 1)°r?c | 


m(2n + 1) 4(b — a)? 


Step 5: Invert U(x, s) term by term and show that 


O 4NA _ [@n+1)r(z-a)] f° (2n + 1)re(t — rT) 
u(x,t) = = Dal 0a) | S acos 30a) dr. 


7. Use transform methods to solve the wave equation 


u u 
ze gaT E 0< z<, 0<t, 


with the boundary conditions u(0,t) = 1 — e™*, limz4.0 |u(z,t)| ~ 2", n finite, 0 < t, and 
the initial conditions u(x,0) = 0, us(x,0) = z, 0 < £ < œ. 

Step 1: Take the Laplace transform of the partial differential equation and boundary con- 
ditions and show that 


dU(z, s8) 


2 
a U(x,5s)=-2- -7, 0< z<, 
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with the boundary conditions 


a) ee 


and lim |U(x, s)| ~ a”. 
S S + £— OO 


Step 2: Show that the solution to the previous step is 


1 1 1 1 tie eon Be 
utes) = (= s+1 | s? zm) ! l ; 


Step 3: Inverting term by term, show that 


u(x,t) = xt — te” + sinh(t)e~” + [1 —e~“-*) + ¢— x — sinh(t — x)| H(t — x). 


8. Use transform methods to solve the wave equation 


u u a4 
= xe 


oe ðe ; OLT LOO; 0<t, 
x 
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with the boundary conditions u(0,t) = cos(t), limgo. |u(z, t)| ~ x”, n finite, 0 < t, and 


the initial conditions u(x,0) = 1, us(£,0) = 0,0 < £z < œ. 


Step 1: Take the Laplace transform of the partial differential equation and boundary con- 


ditions and show that 


dU 
d'U(x,s) _ 0< z<, 


with the boundary conditions U (0, s) = s/(s? +1) and limy-.. |U(z,s)| ~ z”. 


Step 2: Show that the solution to the previous step is 


1 1 
u(e.s) = ( š -i)e ea i 


s 2 s s+4+1 


Step 3: Inverting term by term, show that u(x,t) = 1+st—x+re™*+|cos(t—x)—1]H (t-z). 


9. Use transform methods to solve the wave equation 


u u 


e Ir?’ 0<a<L, 0<t, 


with the boundary conditions 


OPu(L,t k Ou(L,t 
dopo THEN, ROMEO gc 


and the initial conditions u(x, 0) = uz(a,0) = 0,0 < x < L, where k, m, and g are constants. 
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Step 1: Take the Laplace transform of the partial differential equation and boundary con- 
ditions and show that 


dU(z, s8) 


Ti — s°U(z,s)=0, 0<a<L, 


with the boundary conditions U(0,s) = 0 and s?U(L,s) + w?U'(L,s) = g/s, where w? = 
k/m. 
Step 2: Show that the solution to the previous step is 


g sinh(sz) 
s?[s sinh(sL) + w? cosh(sL)] ` 


U(a,s) = 


Step 3: Show that U(x, s) has simple poles at s = 0 and sn = Ani, where Àn = w? cot(A,L) 
with n = 1,2,3,.... 


Step 4: Using Bromwich’s integral, show that 


u(x,t) = 


gr 2gw? 3 sin(A, 2) cos(Ant) 
w? L & Mw + 0?/L + A) sin(anL) 


10. Use transform methods?’ to solve the wave equation 


Pu ð pot 
se ° an\" de 


i O0<a<l, 0<t, 


with the boundary conditions lim,_,9 |u(a,t)| < œ and u(1,t) = Asin(wt), 0 < t, and 
the initial conditions u(z,0) = u(x,0) = 0, 0 < x < 1. Assume that 2w # cßn, where 


Step 1: Take the Laplace transform of the partial differential equation and boundary con- 
ditions and show that 


2 
i [eee : U(x,s)=0, 0<z<1l. 


dx dx ce 
with the boundary conditions lim,_,9 |U (x, s)| < oo and U(1, s) = Aw/(s? + w?). 
Step 2: Show that the solution to the previous step is 


Aw  I9(28./x/c) 
s2 +w? Io(2s/c) 


U(a,s) = 


Step 3: Show that U (x, s) has simple poles at s = +wi and Sn = +c8,i/2, where Jo(8,) = 0, 
n=1,2,3,.. 


23 Suggested by a problem solved by Brown, J., 1975: Stresses in towed cables during re-entry. J. Spacecr. 
Rockets, 12, 524-527. 
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Step 4: Using Bromwich’s integral, show that 


jdol2wVz/c) a tonite 2 Bn VZT ) sin(Bnct/2) 


OSA os eR /4) (Bn) 


11. A lossless transmission line of length @ has a constant voltage E applied to the end 
x = 0 while we insulate the other end [V,(¢,t) = 0]. Find the voltage at any point on the 
line if the initial current and charge are zero. 


Step 1: Take the Laplace transform of the partial differential equation and boundary con- 
ditions and show that 


: 2 
— —U(x,s) = 0, 0<z<f, 
c 


with the boundary conditions U (0, s) = E/s and U’ (£, s) = 0. 
Step 2: Show that the solution to the previous step is 


E cosh[|s(£— x)/c] 
s cosh(st/c) ` 


U(zx,s) = 


Step 3: Show that U(x,s) has simple poles at s = 0 and s, = (2n — 1)cri/(2£) with 
n=1,2,3,... 


Step 4: Using Bromwich’s integral, show that 


ulz) =E 2 > 2 ; sin — pze] cos r Pe 


Step 5: An alternative approach is to replace the hyperbolic functions with their exponential 
definitions. Then, 


U(x, s) = = jee = e7 8(e+26)/c ae e7 8(e+4e)/c Sao | 


E —s(2l—x)/c _ s(4€—x)/c s(6l—2)/c _ |, 
F e e +e 
S 


after using the summation rule for the geometric series. Take the inverse by inspection and 


show that 
=e Sera! aan 3 yali a, 


12. Solve the equation of telegraphy without leakage 


2 2 
aR or O<2<%, 0<t, 
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subject to the boundary conditions u(0, t) = 0, u(é,t) = E, 0 < t, and the initial conditions 
u(x,0) = u,(x,0) =0,0 <x <£. Assume that 4r? L/CR?P > 1. Why? 


Step 1: Take the Laplace transform of the partial differential equation and boundary con- 
ditions and show that 


i 2 
— @U(z,s) =9, 0<2r<%, 


with the boundary conditions U(0,s) = E/s and U’(¢,s) = 0. 
Step 2: Show that the solution to the previous step is 


E cosh[s(@— x)/c] 


U(2,s) = s cosh(sé/c) 


Step 3: Show that U(x,s) has simple poles at s = 0 and sn = +(2n — 1)cmi/(2¢) with 
n=1,2,3,... 


Step 4: Using Bromwich’s integral, show that 


u(z,t) T Ze S E )” D" in (272) j= oo cos(ty n28? — i2r)|. 


13. The pressure and velocity oscillations from water hammer in a pipe without friction?* 
are given by the equations 
Op 2 Ou ae Ou 1 Op 
— =-pc°—, n — =-- =, 
Ot Pe Oe Ot p Ox 
where p(x,t) denotes the pressure perturbation, u(x,t) is the velocity perturbation, c is 
the speed of sound in water, and p is the density of water. These two first-order partial 
differential equations can be combined to yield 
Pp _ 0P 
— = C—, O<a<L, 0<t. 
ot? Ox? 
Find the solution to this partial differential equation if p(0,t) = po, and u(L,t) = 0, and 
the initial conditions are p(x,0) = po, p(x, 0) = 0, and u(az,0) = uo. 


Step 1: Take the Laplace transform of the partial differential equation and boundary con- 
ditions and show that 


d*P(x,s) 3 8 
P(e) _ & ple, 8) = Apo, 0<a<L, 


with the boundary conditions P(0, s) = po/s and P’(L,s) = puo. 
Step 2: Show that the solution to the previous step is 


Po , puoc sinh(sx/c) 
ss cosh(sL/c) 


24 See Rich, G. R., 1945: Water-hammer analysis by the Laplace-Mellin transformation. Trans. ASME, 
67, 361-376. 
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Step 3: Show that P(x,s) has simple poles at sn = +(2n — 1)emi/(2L) with n = 1,2,3,... 
and a removable singularity at s = 0. 


Step 4: Using Bromwich’s integral, show that 


B Apugc ~> (—1)” «| (2n—1)re] . [| (Qn — lent 
P(e, t) = po — = } sin] a sin] | - 


14. Use Laplace transforms to solve the wave equation?" 


Pu (a 20u 2u 


ap = ; <r<oo, 0<t, 
Ot? ðr? or Or = gh ce 


subject to the boundary conditions that 


u(a,t)=A (1 — 2 H(t), lim u(r,t)—> 0, 0<t, 


TOO 
and the initial conditions that u(r,0) = uz(r,0) =0,a<r<o. 
Step 1: Take the Laplace transform of the partial differential equation and boundary con- 


ditions and show that 


@U(r,s) 2dU(r,s) 2 3? 


dr? r dr r2 


with the boundary condition 


A A 
U(a,s) = — — —— lim |U(r,s)| < oo. 


s stc/a’ 7400 


Step 2: Show that the solution to the previous step is 


a? a? c(a a 1 
5% —s(r—a)/c 
U(r, s) É r?(s+c/a) a Ẹ =) ET f 


Step 3: Use tables and the second shifting theorem to show that 


a? a Or fede. aN) ora 
ulrt) = ALS É H a (5 =) (oh HC), 


where T = t — (r—a)/c. 


15. Use Laplace transforms to solve the wave equation?® 


(ru) a 0? (ru) 
Or Or?” 


a<r<ow, 0<t, 


25 Wolf, J. P., and G. R. Darbre, 1986: Time-domain boundary element method in visco-elasticity with 
application to a spherical cavity. Soil Dynam. Earthg. Eng., 5, 138-148. 


26 Originally solved using Fourier transforms by Sharpe, J. A., 1942: The production of elastic waves by 
explosion pressures. I. Theory and empirical field observations. Geophysics, 7, 144-154. 


124 Advanced Engineering Mathematics: A Second Course 


subject to the boundary conditions that 
o(Fu 2 du 
Pe | ar?” 3r Or 


where a > 0, and the initial conditions that u(r, 0) = uz(r,0) =0,a < r < œ. 


= ppe “H(t), lim u(r,t) > 0, 0<t, 
TCO 


r=a 


Step 1: Take the Laplace transform of the partial differential equation and boundary con- 
ditions and show that 
d[rU(r,s)| 3? 
dr? 2 | 


rU (r, s)| = 0, a<r <o. 


with the boundary condition 


d?U(a,s) 2 dU(a,s) Po 
2 ’ | ’ : 
pc 72 ae i ET and jim |U (r, s)| < oo. 


Step 2: Show that the solution to the previous step is 


apo exp|—s(r — a)/c| 
pr(s + a)[s? + 4sc/(3a) + 4c?/(3a?)] 


U(r,s) = 


Step 3: Show that U(r,s) has three simple poles, s = —a and s = —B/\/2 + Bi, where 
B = 2v/2c/(3a). 


Step 4: Use Bromwich’s integral and show that 


where T = t — (r — a)/c. 


16. Consider a vertical rod or column of length L that is supported at both ends. The 
elastic waves that arise when the support at the bottom is suddenly removed are governed 
by the wave equation?” 


Pu Pu 


3p E gaT O O0<a<L, 0<t, 


where g denotes the gravitational acceleration, c? = E/p, E is Young’s modulus, and p is the 
mass density. Find the wave solution if the boundary conditions are u,(0,t) = uz(L,t) = 0, 
0 < t, and the initial conditions are 


gx? Ou(x, 0) 


ulz, 0) = -57> T 


= 0, O<a<L. 


Step 1: Take the Laplace transform of the partial differential equation and boundary con- 
ditions and show that 


@U(x,s) 3? sgx= g 
ar ae 2 = =. 0<a4< L, 


2c4 sc?’ 


27 See Hall, L. H., 1953: Longitudinal vibrations of a vertical column by the method of Laplace transform. 
Am. J. Phys., 21, 287-292. 
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with U’(0,s) = U’ (L, s) = 0. 
Step 2: Show that the solution to the previous step is 


_ gL cosh(sz/c) gz? 
IERIE cs? sinh(sL/c) — 2sc?’ 


Step 3: Show that U(z,s) has poles that are located at s = 0 and sn = tnmci/L, where 
n=1,2,3,.... 


Step 4: Use Bromwich’s integral and show that 


g? gE 06h? (1) NTE nret 
u(x,t) = 5 i= ae 72 cos( 7 ) cos j 
n=1 


17. Use Laplace transforms to solve the hyperbolic equation 


u u 

— -= +1=0 0 1, 0<t 

ar Aa? + 3 <@<l, <t, 
subject to the boundary conditions that u,(0,t) = 0, uz(1,t) = 1, O < t, and the initial 
conditions that u(x,0) = u(x,0)=0,0<x<1. 
Step 1: Take the Laplace transform of the partial differential equation and boundary con- 
ditions and show that 

d U(x, s) 


1 
= — 8'U(z,8)=—+2°—1, 0<a<1, 


with U’(0,s) = 0 and U’(1,s) =1/s. 
Step 2: Show that the solution to the previous step is 


1-2? 1 2 cosh(sx) — 2cosh(sz) 
U(x, 8) = 3 3 T+ ag + ai : 
s s S s? sinh(s) s3 sinh(s) 
Step 3: Show that U(z,s) has poles that are located at s = 0 and zn = +nri, where 


AOAR 
Step 4: Use Bromwich’s integral and show that 


Aeta = " cos(nmt)  2sin(nzt) 
u(x,t) = a a g 22 1)” cos(nrzx) | na? 373 
18. Solve the telegraph-like equation?® 
o?u Es ðu 4 (Au Ou jee oy 
— —=c (=> — z 
ae |” at ðr? | Ox) ae hee 


28 See Abbott, M. R., 1959: The downstream effect of closing a barrier across an estuary with particular 
reference to the Thames. Proc. R. Soc. London, Ser. A, 251, 426-439. 
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subject to the boundary conditions u,(0,t) = —uod(t), limz.. u(x,t) > 0, 0 < t, and the 
initial conditions u(x, 0) = uo, w(x, 0) = 0, 0 < x < œ, with ac > k. Here 6(t) denotes the 
Dirac delta function. 


Step 1: Take the Laplace transform of the partial differential equation and boundary con- 
ditions and show that 


2 2 
d*U(z, s) „aU E9) sf + ks es s+k t (Ses: 
da? dx e c? 


with U’ (0, s) = —uo, and lims U(x, s) > 0. 


Step 2: Show that the solution to the previous step is 
exp E (s + k)? + a/e! 


U(a,s) = Hoj uge™®?/? : 
° F +y (s +3)? +a?/c 


where 4a? = a?c? — k? > 0. 


Step 3: Using the first and second shifting theorems and the property that 


F(vs? +a?) -c| af A ra 
0 


VE — 72 
show that 


J (avt — 77) 


t2 — 72 


t 
u(x,t) = uo + uoce ™™/? H(t — z/c) a af reread] ; 
a/c 


19. As an electric locomotive travels down a track at the speed V, the pantograph (the 
metallic framework that connects the overhead power lines to the locomotive) pushes up 
the line with a force P. Let us find the behavior”? of the overhead wire as a pantograph 
passes between two supports of the electrical cable that are located a distance L apart. We 
model this system as a vibrating string with a point load: 


a l Ox? us pV 


2 2 
P 
aP E a(t Z), Oa LE, 0<t. 


Let us assume that the wire is initially at rest [u(#,0) = u¢(x,0) = 0 for 0 < x < L] and 
fixed at both ends [u(0,t) = u(L,t) = 0 for 0 < t]. 


Step 1: Take the Laplace transform of the partial differential equation and show that 


2@U(z, 8) q E e-es/V 


2 = 
s “U(x,s) =c Ta BV ; 


O0<a<L. 


29 See Oda, O., and Y. Ooura, 1976: Vibrations of catenary overhead wire. Q. Rep., (Tokyo) Railway 
Tech. Res. Inst., 17, 134-135. 
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Step 2: Solve the ordinary differential equation in Step 1 as a Fourier half-range sine series 


= Yo Bale )sin( 2 Z) 7 


where 


= ZP Br 1 1 ncbi 
TIS T= a) Fee =| |1- cre], 


an = nrc/L and 8, = nt V/L. This solution satisfies the boundary conditions. 


Step 3: By inverting the solution in Step 2, show that 
sin mee V sin(ant)] . eE 
wt | sin 
c an — Br L 


E Le 1ytsin( 272) flet 1 V inlet AN, 


cam L — ß2 c a? — 82 


or 


c a, — BR 


B E 7 n a sin(™2) (5 K 1)" met \ 


n=1 


_ 2P > eee Bnt) V | sin( 277) 


The first term in both summations represents the static uplift on the line; this term dis- 
appears after the pantograph passes. The second term in both summations represents the 
vibrations excited by the traveling force. Even after the pantograph passes, they continue 
to exist. 


20. Solve the wave equation 


18u u lôu u (r-a) 
= 0< 0<t 
c2 ôt? ðr? Ror e ae ee =e 
where 0 < a < a, subject to the boundary conditions lim,_,9 |u(r,t)|_ < oo, u,(a,t) + 
hu(a,t)/a = 0, 0 < t, and the initial conditions u(r,0) = u:(r,0) =0,0<r<a. 
Step 1: Take the Laplace transform of the partial differential equation and show that 
2 2 1 = 
d°U(r,s) | 1dU(r,s) cae tC a) eee 
dr? r dr 2 


with lim,_,9 |U (r, s)| < co and U’ (a, s) + ŁU (a, s) = 0. 
Step 2: Show that the Dirac delta function can be reexpressed as the Fourier-Bessel series 


Co 


= pe Ji (Bna/a) 
a? (62 + h? —1)J? (Bn) 


where Bn is the nth root of BJ{(B) +h Ji(8) = BJo(B) + (h — 1)Ji(8) = 0 and Jo(-), Ji (-) 
are the zeroth and first-order Bessel functions of the first kind, respectively. 


Ji(Bnr/a), O0O<r<a, 
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Step 3: Show that the solution to the ordinary differential equation in Step 1 is 


Hee 2 ys L f s | 


a L (B3 +h? — 1) J? (Ba) s? + BR /a? 


Note that this solution satisfies the boundary conditions. 


Step 4: Taking the inverse of the Laplace transform in Step 3, show that the solution to 
the partial differential equation is 


a E 


21. Solve the hyperbolic equation 


8u 
say t=O 0< 2,t, 


subject to the boundary conditions u(0, t) = e~*, lims u(x,t) > 0,0 < t, and u(x, 0) = 1, 
limo lulz, t)| < Me**, 0 < k, M,2,t. 
Step 1: Take the Laplace transform of the partial differential equation and show that 
dU 
(dU (e,s) 


+U =0, 0< z<, 
dx 


with U(0,s) = 1/(s + 1) and limy_,~ U (z, s) > 0. 
Step 2: Show that 


Step 3: Using tables and the convolution theorem, show that the solution is 


t 
u(x,t) = Jo(2V at) — D e7” Jo(2\/aT ) dr 
0 
where Jo(-) is the Bessel function of the first kind and order zero. 


22. Solve the hyperbolic equation 


O2u Ou Ou 
| pts 
pedi oe OE DES lt 


subject to the boundary conditions u(0,t) = e*, limy +o u(x,t) > 0, 0 < t, and the initial 
conditions u(x,0) = 1, limy-s. |u(z, t)| < Me**, 0 < k, M,t, x. 


Step 1: Take the Laplace transform of the partial differential equation and show that 


d 
e+ OED) + as =a, 0< z<, 


with U (0, s) = 1/(s — c) and limy_,. U (z, s) > 0. 
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Step 2: Show that 


1 ce * ba 
U(a,s) = EA ep( 2) i 


Step 3: Using tables, the first shifting theorem, and the convolution theorem, show that 
the solution is 


u(z,t) = 1 + ce” i eT +e)" J, (2vber ) dr, 
where Io(-) is the modified Bessel function of the first kind and order zero. 
2.5 THE SOLUTION OF THE HEAT EQUATION BY USING LAPLACE TRANSFORMS 
In the previous section we showed that we can solve the wave equation by the method 


of Laplace transforms. This is also true for the heat equation. Once again, we take the 
Laplace transform with respect to time. From the definition of Laplace transforms, 


Liu(a, t)] = U(x, s), (2.5.1) 
Liuz(a,t)] = sU (zx, s) — u(a, 0), (2.5.2) 

and sip 
L[uUae (a, t)] = e (2.5.3) 


We next solve the resulting ordinary differential equation, known as the auxiliary equation, 
along with the corresponding Laplace transformed boundary conditions. The initial condi- 
tion gives us the value of u(x,0). The final step is the inversion of the Laplace transform 
U(x,s). We typically use the inversion integral. 


e Example 2.5.1 


To illustrate these concepts, we solve a heat conduction problem®? in a plane slab of 
thickness 2L. Initially the slab has a constant temperature of unity. For 0 < t, we allow 
both faces of the slab to radiatively cool in a medium that has a temperature of zero. 

If u(x,t) denotes the temperature, a? is the thermal diffusivity, h is the relative emis- 
sivity, t is the time, and z is the distance perpendicular to the face of the slab and measured 
from the middle of the slab, then the governing equation is 


Ou 2074 
a T Bq? L<xr<L, 0<t, (2.5.4) 
with the initial condition 
u(x,0) = 1, -L<a<lL, (2.5.5) 
and boundary conditions 
Ou(L, t Ou(—L, t 
OUD). hu(L,t) =0, USED hu(—L,t)=0, 0<t. (2.5.6) 
Ox Ox 


30 Goldstein, S., 1932: The application of Heaviside’s operational method to the solution of a problem 
in heat conduction. Z. Angew. Math. Mech., 12, 234-243. 
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Taking the Laplace transform of Equation 2.5.4 and substituting the initial condition, 


dU 
eeo — sU (x, s) = —1. (2.5.7) 
If we write s = a?q?, Equation 2.5.7 becomes 
dU (x, s8) 1 


From the boundary conditions, U(x, s) is an even function in z and we may conveniently 
write the solution as 


U(x,s) = i + Acosh(qz). (2.5.9) 
From Equation 2.5.6, 
gAsinh(qL) + " + hAcosh(qL) = 0, (2.5.10) 
and 
Ce neceh(an) (2.5.11) 


~ 5 s{qsinh(qL) + hcosh(qL)]’ 


The inverse of U(x,s) consists of two terms. The first term is simply unity. We will 
invert the second term by contour integration. 

We begin by examining the nature and location of the singularities in the second term. 
Using the product formulas for the hyperbolic cosine and sine functions, the second term 
equals 


n (1+) (1+) s 


sjer (1+ 2E) (1 tin)... n(1+ 427) (1 ge)... 


(2.5.12) 


Because q? = s/a?, Equation 2.5.12 shows that we do not have any ys in the transform 
and we need not concern ourselves with branch points and cuts. Furthermore, we have only 
simple poles: one located at s = 0 and the others where 

qsinh(qL) + hcosh(qL) = 0. (2.5.13) 


If we set q = ià, Equation 2.5.13 becomes 


heos(AL) — Asin(AL) = 0, or AL tan(AL) = hL. (2.5.14) 


From Bromwich’s integral, 


= hcosh(qa) 1 ¢ hcosh(qax)e* 
1 — 
£ ‘ai sinh(qL) + ET) — Oni Jo zlqsinh(qL) + hcosh(qL)| dz, (2.5.15) 
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U(X,T) 


0.6 
gaan? 


Figure 2.5.1: The temperature within the portion of a slab 0 < z/L < 1 at various times a?t/L? if the 
faces of the slab radiate to free space at temperature zero and the slab initially has the temperature 1. The 


parameter hL = 1. 


where q = z1/? /a and the closed contour C consists of Bromwich’s contour plus a semicircle 


of infinite radius in the left half of the z-plane. The residue at z = 0 is 1 while at zn = —a? A2, 
Re hcosh(qx)e* TE Met a? A2) cosh(qa)e’* (2.5.16) 
z{qsinh(qL) + hcosh(qL)] z>zn z[qsinh(qL) + hcosh(qL)] 
— lim hcosh(qx)e** 
2-42, 2[(1 + AL) sinh(qL) + qL cosh(qL)]/(2a2q) 


(2.5.17) 
2ha? Ani cosh(iA, £) exp(—A?.a7t) 
Pe Wain aL) Pa A] 
(2.5.18) 


2h cos(Anx) exp(—a?A2t) 
An[(1 + AL) sin(A,L) + àn L cos(AnL)] | 
(2.5.19) 


Therefore, the inversion of U (x, s) is 


u(z,t) =1— fı Dh 3 5 bos nt a a and) \ (2.5.20) 


n[(1 + AL) sin(AnL) + AnL cos(A,L)] 


(2.5.21) 


cos(Àn g) exp(—a?A2t) 
t) = 2h 
ae 3 Xnl(l + AL) sin(An L) + AnL cos(AnL)] 


We can further simplify ee 2.5.21 by using h/An = tan(A,L). This yields hL = 
AnL tan(A,L). Substituting these relationships into Equation 2.5.21 and simplifying, 


sin(ÀAn L) cos(Anx) exp(—a? A2 t) 
=2 : 2.5.22 
wed) 3 An L + sin(An L) cos(An L) (aa 


Figure 2.5.1 illustrates Equation i It was created using the MATLAB script 
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clear 
hL = 1; m = 0; M = 100; dx = 0.05; dt = 0.05; 
% create initial guess at zeron 
zero = zeros(length(M)) ; 
for n = 1:10000 
ki = 0.1*n; k2 = 0.1*(nt1); 
prod = ki * tan(ki); y1 = hL - prod; y2 = hL - k2 * tan(k2); 
if (yl*y2 <= 0 & prod < 2 & m < M) m = m+1; zero(m) = ki; end; 
end; 
% use Newton-Raphson method to improve values of zeron 
for n = 1:M; for k = 1:10 
f = hL - zero(n) * tan(zero(n)); 


fp = - tan(zero(n)) - zero(n) * sec(zero(n))*2; 
zero(n) = zero(n) - f / fp; 
end; end; 


% compute Fourier coefficients 
for m = 1:M 
a(m) = 2 * sin(zero(m)) / (zero(m) + sin(zero(m))*cos(zero(m))); 
end 
% compute grid and initialize solution 
[O:dx:1]; T = [0:dt:2]; 
u = zeros(length(T) , length(X)) ; 
XX = repmat(X,[length(T) 1]); TT = repmat(T’,[1 length(X)]); 
% compute solution from Equation 2.5.22 
for m = 1:M 
u =u + a(m) * cos(zero(m)*XX) .* exp(-zero(m)*zero(m)*TT) ; 
end 
surf (XX,TT,u) 
xlabel(’distance’,’Fontsize’,20); ylabel(’time’,’Fontsize’ ,20) 
zlabel(’?U(X,T)’,’Fontsize’ ,20) 


es 
ul 


e Example 2.5.2: Heat dissipation in disc brakes 


Disc brakes consist of two blocks of frictional material known as pads that press against 
each side of a rotating annulus, usually made of a ferrous material. In this problem we deter- 
mine the transient temperatures reached in a disc brake during a single brake application.*! 
If we ignore the errors introduced by replacing the cylindrical portion of the drum by a 
rectangular plate, we can model our disc brakes as a one-dimensional solid, which friction 
heats at both ends. Assuming symmetry about x = 0, the boundary condition there is 
u,(0,t) = 0. To model the heat flux from the pads, we assume a uniform disc deceleration 
that generates heat from the frictional surfaces at the rate N(1— Mt), where M and N are 
experimentally determined constants. 

If u(x,t), k, and a? denote the temperature, thermal conductivity, and diffusivity of 
the rotating annulus, respectively, then the heat equation is 

ðu u 


= aí — L t 2.5.23 
aE Bee O0<a<L, 0<t, (2.5.23) 


31 From Newcomb, T. P., 1958: The flow of heat in a parallel-faced infinite solid. Brit. J. Appl. Phys., 
9, 370-372. See also Newcomb, T. P., 1958/59: Transient temperatures in brake drums and linings. Proc. 
Inst. Mech. Eng., Auto. Div., 227-237; Newcomb, T. P., 1959: Transient temperatures attained in disk 
brakes. Brit. J. Appl. Phys., 10, 339-340. 
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with the boundary conditions 


Ou. t) _ 


N(1- Mb), 0<t. (2.5.24) 
Ox 


The boundary condition at x = L gives the frictional heating of the disc pads. 
Introducing the Laplace transform of u(x,t), defined as 


U(x, 8) = A u(x,t) dt, (2.5.25) 


the equation to be solved becomes 


dU s 


subject to the boundary conditions that 


dU(0,s) _ dU(L,s) N(1 M 
=A =U, and he = fae he z2 (2.5.27) 
The solution of Equation 2.5.26 is 
U(a,s) = Acosh(qx) + Bsinh(qz), (2.5.28) 
where q = st/? /a. Using the boundary conditions, the solution becomes 
N/1  M\ cosh(qz) 
U = : 2.5.29 
oe) K G s? ) qsinh(qL) ( ) 


It now remains to invert the transform, Equation 2.5.29. We will invert cosh(qz)/ 
[sqsinh(qL)]; the inversion of the second term follows by analog. 

Our first concern is the presence of s!/? because this is a multivalued function. However, 
when we replace the hyperbolic cosine and sine functions with their Taylor expansions, 
cosh(qx)/[sqsinh(qL)| contains only powers of s and is, in fact, a single-valued function. 

From Bromwich’s integral, 


c+toot tz 
ra cosh(qa) |- 1 / cosh(qx)e (2.5.30) 


sqsinh(qL) Q7t Jeon; zqsinh(qL) 
where q = 2"? /a. Just as in the previous example, we replace the hyperbolic cosine and 
sine with their product expansion to determine the nature of the singularities. The point 
z = 0 is a second-order pole. The remaining poles are located where zal "Ly a = nmi, or 
Zn = —n?n?a?/L?, where n = 1,2,3,.... We have chosen the positive sign because z1/? 
must be single-valued; if we had chosen the negative sign, the answer would have been the 
same. Our expansion also shows that the poles are simple. 

Having classified the poles, we now close Bromwich’s contour, which lies slightly to the 
right of the imaginary axis, with an infinite semicircle in the left half-plane, and use the 
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residue theorem. The values of the residues are 


z tz _ 2 n tz 
M coahige)¢ ol = 1 lim d {(z-0) SOn ae (2.5.31) 
zqsinh(qL) 1! 250 dz zqsinh(qL) 
. df zcosh(qx)e* 
=] 2.5.32 
240 dz { qsinh(qL) Poe) 
za? t?z? 
-Fim f z+ Ee pee. \ 2088) 
a a er i l (2.5.34) 
~ T 240 dz ZT Da? 3a? i 
a? zr? L? 
= t 2.5.35 
L { * 9a? Ga? \. ( ) 
and 
sh t h — 2n, 
es DUE g, = | lim cose) i lim eee ee (2.5.36) 
zqsinh(qL) Zin = zq z>zn sinh(qL) 
; cosh(qx)e!* 
= 2.5.3 
atin zq cosh(qL)L/(2a?q) (218.37) 
_ cosh(naai/L) exp(—n?1?a7t/L?) (2.5.38) 
-© (—=n?r?a?/L?) cosh(nmi)L/ (2a?) n 
2L(—1)” —n? na? 3 
D cos(nrz/L)e HE (2.5.39) 
When we sum all of the residues from both inversions, the solution is 
a? N 2 rP 2LN & (-1)" n2n2a?t/ L? 
u(x,t) = T fi- 242 aa} am? u cos(n7a/L)e / 
a NM { t? tz? tL? ri aD? l 7L 
kL |2 ` 2a? 6a? ` 24a4 12at ` 36004 
2L°NM & (-1)" 
a = cos(nre/L)e tE. (2.5.40) 
n=1 


Figure 2.3.2 shows the temperature in the brake lining at various places within the 
lining |z" = x/L] if a? = 3.3 x 10-3 cm?/sec, k = 1.8 x 107? cal/(cm sec°C), L = 0.48 cm, 
and N = 1.96 cal/(cm? sec). Initially the frictional heating results in an increase in the disc 
brake’s temperature. As time increases, the heating rate decreases and radiative cooling 
becomes sufficiently large that the temperature begins to fall. 


e Example 2.5.3 


In the previous example we showed that Laplace transforms are particularly useful 
when the boundary conditions are time dependent. Consider now the case when one of the 
boundaries is moving. 
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80 


TEMPERATURE (°C) 


TIME (SECONDS) 


Figure 2.5.2: Typical curves of transient temperature at different locations in a brake lining. Circles 
denote computed values while squares are experimental measurements. (From Newcomb, T. P., 1958: The 
flow of heat in a parallel-faced infinite solid. Brit. J. Appl. Phys., 9, 372 with permission.) 


We wish to solve the heat equation 


Ou 20u 
mw Ba Bt<au<ow, 0<t, (2.5.41) 
subject to the boundary conditions 
u(x, t)| ap =f(t), and lim u(a,t)>0, 0<t, (2.5.42) 
and the initial condition 
u(x, 0) = 0, 0< z<. (2.5.43) 


This type of problem arises in combustion problems where the boundary moves due to the 
burning of the fuel. 

We begin by introducing the coordinate 7 = x — Gt. Then the problem can be refor- 
mulated as 


Ou Ou 20u 


= 0 0<t 2.5.44 
p Pan Oe” <<, A ( ) 
subject to the boundary conditions 
u(0,t) = f(t), lim u(n,t) > 0, 0<t, (2.5.45) 
n—> co 


and the initial condition 
u(n, 0) = 0, 0<n<m. (2.5.46) 
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Taking the Laplace transform of Equation 2.5.44, we have that 


@U(n,s) B dU(n,s) s 
dn? i 2 


U(n,s) = 0, (2.5.47) 
a dn a 


with 
U(0,s) = F(s), and lim U(n, s) > 0. (2.5.48) 
Noo 


The solution to Equation 2.5.47 and Equation 2.5.48 is 


Bn n p? 
=f } 2.5.49 
U(n,s) = F(s) oo( P hjst f (2.5.49) 
Because 
= n p? 
L[®(n, t)] = exp| —-1/s + — ], (2.5.50) 
a 4a? 
where 
= t 2 n , Bvt 
(n,t)= de anea 4 et) H ef"/2a erfe( 2 + I) 2.5.51 
(nt) =3| 5 +e (2.5.51) 
and j ; 
erfe(x) = 1 — = | e-” dn, (2.5.52) 
VT Jo 
we have by the convolution theorem that 
> rt 
u(n,t) = e78n/2a I f(t — r)®(n,rT)dr, (2.5.53) 
0 
or ; 
u(x,t) = e Ble— At) /20° J f(t—7)®(a — BrT, T) dr. (2.5.54) 
0 
Problems 
1. Solve the heat equation 
ðu u , 
a om a“ (u — To), 0<xr<1, 0<t, 


subject to the boundary conditions uz(0,t) = uz(1,t) = 0, 0 < t, and the initial condition 
u(z,0) =0,0<a<1. 


Step 1: Take the Laplace transform of the partial differential equation and boundary con- 
ditions and show that 


a? To 


2 
L ae EE = 0<a<l, 


dx? 


subject to the boundary conditions U’(0,s) = U'(1,s) = 0. 
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Step 2: Show that the solution to the previous step is 


Step 8: Invert U(x, s) and show that u(x,t) = To (1 — eae), 


2. Solve 5 22 
u u 

— = — 0 1, 0<t 

ðt = Ox?’ a a a 
subject to the boundary conditions u,(0,¢) = 0, u(1,t) = t, 0 < t, and the initial condition 
u(z,0) =0,0<a<1. 
Step 1: Take the Laplace transform of the partial differential equation and boundary con- 
ditions and show that 


aU 
TTS) L sy(a,s) =0, 0<2<1, 
subject to the boundary conditions U’(0,s) = 0 and U(1, s) = 1/s?. 


Step 2: Show that the solution to the previous step is 


_ cosh(x7/s ) 
U(x, s) = Soma). 


Step 3: Show that U(z,s) has a second-order pole at s = 0 and simple poles at sn = 
—(2n — 1)?n?/4 with \/z, = (2n — 1)ri/2, where n = 1,2,3,.... 
Step 4: Use Bromwich’s integral and show that 


a 2 se cos| 2 =H] a| (2n — a l 


u(x,t) =t+ La? 1) 


3. Solve 7 32 
u u 

— = — 0 1, O<t 

at J2’ LTK <t, 
subject to the boundary conditions u(0,t) = 0, u(1,t) = 1, 0 < t, and the initial condition 
u(z,0) =0,0<a<1. 
Step 1: Take the Laplace transform of the partial differential equation and boundary con- 
ditions and show that 


dU(z, 8) 
a SU (2, 5) = 0, 0<a<il, 
subject to the boundary conditions U(0,s) = 0 and U(1,s) =1/s. 

Step 2: Show that the solution to the previous step is U(x, s) = sinh(xvys ) /[s sinh(¥/s )]. 


Step 3: Show that U(a,s) has simple poles at s = 0 and sn = —n?x? with /Z, = nri, 
where n = 1,2,3,.... 
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Step 4: Use Bromwich’s integral and show that 


2 (-1)".. n?n? 
t — — WEN . 
u(x,t) =x + z ) sin(n7x)e 


4. Solve 

ðu Pu 

ðt Oa?’ 
subject to the boundary conditions Ue (—$,t) = 0, Ux (5, t) = 6(t), O < t, and the initial 
condition u(x,0) = 0, —4 < x < 4. Here 6(t) is the Dirac delta function. 


i 1 
7 << 5 0<t, 


Step 1: Take the Laplace transform of the partial differential equation and boundary con- 
ditions and show that 


= 


dU (x, 8) 


1 
qe — sU(x,s) =0, “5 SBS 


2 
subject to the boundary conditions U'(-s, s) = 0 and U'(s, s) =f: 
Step 2: Show that the solution to the previous step is 


cosh| (a + 4)vs] 


Caa 


Step 3: Replacing the hyperbolic functions by their exponential definition, show that 


U (as) = J {exp|(e— $) v5] +exp[- (e+ §) v51} (Ibe et +). 


Step 4: Taking the inverse of U(x, s) term by term, show that 


1 OO 
u(x,t) = —— exp 
vat i 


Step 5: Show that U(a,s) has simple poles at s = 0 and sn = —n?r? with /Z, = nri, 
where n = 1, 2,3,.... 


Step 6: Use Bromwich’s integral and show that 


2; 2 


u(x,t) =1+ 2X (-1)" cos[nm («+ 4)| e7” 7 *. 


5. Solve Š 52 
U u 
— — — = 1 0 1, O<t 
at 3x2 i <“<l, <t, 
subject to the boundary conditions u(0,t) = u(1,t) = 0, 0 < t, and the initial condition 


u(x,0)=0,0<7z<1. 
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Step 1: Take the Laplace transform of the partial differential equation and boundary con- 
ditions and show that 


dU (a, s) 


dx? 


subject to the boundary conditions U(0,s) = U(1,s) = 0. 


1 
— sU (x, s) = — =, oere h 
s 


Step 2: Show that the solution to the previous step is 


ala cosh(xys) [1 — cosh(vs )] sinh(xys ) 
s? s2? sinh(/s ) 


Step 3: Show that U(x, s) has a second-order pole at s = 0 and simple poles at sn = —n?7? 
with ,/z, = nai, where n = 1,2,3,.... 


Step 4: Use Bromwich’s integral and show that 


Mea E x(1— zx) Aes sin|(2m — 1)rz] -(2m—1)?n?t 
2 T3 om (2m — 1)8 
6. Solve’? : és 
u 9 0°u 
a gee 0<4%< mM, 0<t, 


subject to the boundary conditions u(0,t) = 1, limz-... u(x,t) > 0, 0 < t, and the initial 
condition u(x,0) =0,0<a< oo. 


Step 1: Take the Laplace transform of the partial differential equation and boundary con- 
ditions and show that 


dU(x,s) s8 

— ` gs) =O 0< z<, 
subject to the boundary conditions U (0, s) = 1/s and limz_,. |U (x, s)| < oo. 
Step 2: Show that the solution to the previous step is U(x, s) = e~*V*/4/s. 


Step 3: From an extensive table of inverses, show that u(x,t) = erfc[|x/(2avt)]|, where 
erfc(-) is the complementary error function. 


7. Solve 
ðu Pu 
Ot Ox?’ 
subject to the boundary conditions u,(0,t) = 1, limz+. u(#,t) > 0, 0 < t, and the initial 
condition u(#,0) =0,0<a< oo. 


O0<xr<ow, 0<t, 


Step 1: Take the Laplace transform of the partial differential equation and boundary con- 
ditions and show that 


d'U (za, s) 


ia — sU (x, s) = 0, 0< z<, 
z 


32 Tf u(x,t) denotes the Eulerian velocity of a viscous fluid in the half space x > 0 and parallel to the wall 
located at x = 0, then this problem was first solved by Stokes, G. G., 1850: On the effect of the internal 
friction of fluids on the motions of pendulums. Proc. Cambridge Philos. Soc., 9, Part II, [8]—[106]. 
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subject to the boundary conditions U’(0,s) = 1/s and lim,_,.. |U (x, s)| < oo. 
Step 2: Show that the solution to the previous step is U(x, s) = —e~*V*/s9/?, 


Step 3: From an extensive table of inverses, show that 


ulet) = werte( 27) 2] £ exn( =). 


where erfc(-) is the complementary error function. 


8. Solve 

a = oe O0<x<o, O<t 

Ot = Ox?’ : . 
subject to the boundary conditions u(0,t) = 1, lim,_... u(x,t) > 0, 0 < t, and the initial 
condition u(z,0) = e7”, 0 < £ < œ. 


Step 1: Take the Laplace transform of the partial differential equation and boundary con- 
ditions and show that 


@U (x, s) 


da — sU (x, s) = —e 7, 0< z<, 
z 


subject to the boundary conditions U (0, s) = 1/s and limz_,. |U (x, s)| < co. 


Step 2: Show that the solution to the previous step is 


an 1 1 
— —ay/s 
U(x, s) =l L) ; 


Step 3: From an extensive table of inverses, show that 


u(x,t) = e? + erte 5) — jet jerrerte( =, = vi) + earte( 55 + vi) , 


where erfc(-) is the complementary error function. 


9. Solve 


ðu [9u o nôu 
anh aaa + du) 0<xr<w, 0<t, 


where 6 is a constant, subject to the boundary conditions u(0, t) = uo, limz—o0 u(x, t) > 0, 


0 < t, and the initial condition u(x, 0) = 0, 0 < a < oo. 


Step 1: Take the Laplace transform of the partial differential equation and boundary con- 
ditions and show that 


U(x, s) 


dx +(1+60) 


dU (a, s) s 7 
Tz + (o Z) U(e,s) =0, 0<zr<, 


subject to the boundary conditions U (0, s) = uo/s and limy-4. |U (x, s)| < œœ. 


Step 2: Show that the solution to the previous step is 


(1+8) x /a?(1—6)? 


2 a 4 e|: 


U(x,s) = a exp 
s 
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Step 3: From an extensive table of inverses, show that 


y= wo ete mace i") +2 etorte = a(1 <u) | 


10. During their modeling of a chemical reaction with a back reaction, Agmon et al.33 


solved 
ðu Fu 


L ee 
ot Ox? 
subject to the boundary conditions 


0<x4r<w, 0<t, 


«w+ 00O 


t 
ka + a°uz(0, t) + na | Uz0, T)dr = kru(0,t), lim u(x,t) > 0, 0<t, 
0 


and the initial condition u(z,0) = 0, 0 < x < oo, where ka and «, denote the intrinsic 
dissociation and recombination rate coefficients, respectively. What should they have found? 


Step 1: Take the Laplace transform of the partial differential equation and boundary con- 
ditions and show that 


dU 
TTR) _ SUle,s)=0, 0<2r< um, 


subject to the boundary conditions lim,_,,, |U (x, s)| < co and 


ka + (s + ka)a? U’ (0, s) = sk,U (0, s). 


Step 2: Show that the solution to the previous step is 


uE 2ka exp(—a2y/s/a) 1 1 
F aAvys 2ay/s +kr— A^ ays +kr +A)’ 


where A = y k2 — 4a?rka. 


Step 3: From an extensive table of inverses, show that 


u(x,t) = OS eee le erfc(x_) — eterfe(x+)| ; 


where z+ = |x + (xr + A)a?t]/(2avt) and erfe(-) is the complementary error function. 
11. Solve*# E 
u u 
J pz ÊY 0< z<, 0<t, 


,t)—uz(0, t) = el? -0t limr—o u(x,t) 3 0,0 <t, 
< x < œ, where 8, p, and o are constants and 


subject to the boundary conditions pu( 
and the initial condition u(z,0) = 0, 


o +p. 


0 
0 


33 Agmon, N., E. Pines, and D. Huppert, 1988: Germinate recombination in proton-transfer reactions. 
II. Comparison of diffusional and kinetic schemes. J. Chem. Phys., 88, 5631-5638. 


34 Saidel, G. M., E. D. Morris, and G. M. Chisolm, 1987: Transport of macromolecules in arterial wall 
in vivo: A mathematical model and analytic solutions. Bull. Math. Biol., 49, 153-169. 
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Step 1: Take the Laplace transform of the partial differential equation and boundary con- 


ditions and show that 
aU 
CUS) _ (5 + BU (e,8) =, 0< z<, 


subject to the boundary conditions 


1 


—_ / = —— 
pU(0.8) — 0"(0.8) = —a—. 


Jim |U(a, s)| < 0S; 


Step 2: Show that the solution to the previous step is 


E exp(—-rvVs +8) 
(s+B-—o0?)(p+Vs+B) 


Step 3: Using partial fractions, show that 


(2,5) 2 — 
U(x,s) = = — 
(s)+.0?)(Vs' +p) (v's! Foys =a + p) 
e72y s en evs en ave 


(P-V +p) 


where s’=s+4 8. 


2a(p+a)(vs'—c) 2a(p—0)(Vs' +0)’ 


Step 4: Using the first shifting theorem and the fact that 


—k/s 1 k2 k 
—1 e ak ,a7t 
= f t + —- 
( Z) Srt ep( z) ae e er e(avi 5) 5 


show that 

2 eo” x e°* x 

u(x,t) = de” ve | erfe( o i) | ente( | avi) 
(Haa pto 2/t p-o 24/t 
P px+p°t-6t T 
— e erfe| — + pvt). 
P-o? & £ ) 
12. Solve 5 m 
u ot 2 u —ka 
BE a za t 4e ‘ 0<r<w, 0<t, 


subject to the boundary conditions u,(0,t) = 0, lim: u(x, t) = uo, 0 < t, and the initial 
condition u(a,0) = uo, 0 < £ < oo. 


Step 1: Take the Laplace transform of the partial differential equation and boundary con- 
ditions and show that 


d@U(a,s) s uo Á aie 
— pl 9) =-a- ae s 0< z<, 


subject to the boundary conditions U’ (0, s) = 0 and lim,-,~ U (x, s) = uo/s. 
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Step 2: Show that the solution to the previous step is 


ee uo x Ae hk 1 1 = Ae a Ae~@* 
A p — 
ee s atk? \s—a?k? s aksy/s akys(s — a2k?)’ 


where q = /s/a. 
Step 3: Using the convolution theorem, 


Ae—k* a?k?t A 
seams t(n) 


_ Aew Ft fete 7 x? dr 
ak Jo 4a?T) YTT 


13. Solve 

— =a —f, O0<a<L, 0<t, 
subject to the boundary conditions u(0,t) = t, u(L,t) = 0, 0 < t, and the initial condition 
u(z,0) =0,0<a<L. 


Step 1: Take the Laplace transform of the partial differential equation and boundary con- 
ditions and show that 


@U(z,8), s P 
ae Gn oea 0<a<L, 


subject to the boundary conditions U(0,s) = 1/s? and U(L, s) = 0. 


Step 2: Show that the solution to the previous step is 


P { sinh(qx) A _ „Sinh[g(L — x)] 
z ae 1 re s? sinh(qL) ” 


U(x,s) = 


where q = \/s/a. 
Step 3: Show that U(z,s) has a second-order pole at s = 0 and simple poles at sn = 
—n?r?a? /L? and ./s, = nrai/L, where n = 1,2,3,.... 
Step 4: Use Bromwich’s integral and show that 
t(L — 2) i Pz(x— L) «(a —L)(a -2L) 

L 2a? 6a? L 
2PL? = (-1)" . sna ( ao 

sin( 7 ) exp 


a2r3 n3 L2 
n=1 


u(x,t) = 


Co 


aE +1)? 1 . ¢nra an? nt 
i arn? > n3 sin( L ) me L? f 


14. Solve 
Ou 20u 


EE age ee L k 
OE aaa + hu, O<a<L, 0<k,t, 
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subject to the boundary conditions u(0,t) = u(L,t) = To, 0 < t, and the initial condition 
u(z,0)=T,0<a<L. 


Step 1: Take the Laplace transform of the partial differential equation and boundary con- 
ditions and show that 


@U(z,s) s—k To 
da P U(x,s) = -73 0<a<L, 


subject to the boundary conditions U(0,s) = U (L, s) = To/s. 
Step 2: Show that the solution to the previous step is 


To kTo  sinh(qa) + sinh[q(L — x)| 
s—k  s(s—k) sinh(qL) l 
where q = vs — k/a. 


Step 3: Show that U(x, s) has simple poles at s = 0, s = k, and sn = k — n?nr?a?/L?, where 
n = 1,2,3,.... Note here that qn = nri/L. 


Step 4: Use Bromwich’s integral and show that 


= To cos|(L/2 — x),/k/a?] 
cos(Ly/k/a? /2) 


_ 4kTo 5 sin[(2m — 1)rz/L] okt- (2m—1)?7?a?t/ 1? 
T (2m — 1)[k — (2m — 1)?r?a?/L?] 


u(a, t) 


m=1 


_ ATo se 1 Kin k ree E (2m — 1)ra 
© r Daal EDL a L , 


where km = (2m — 1)?7?a?/L?. 


15. An electric fuse protects electrical devices by using resistance heating to melt an en- 
closed wire when excessive current passes through it. A knowledge of the distribution of 
temperature along the wire is important in the design of the fuse. If the temperature rises 
to the melting point only over a small interval of the element, the melt will produce a 
small gap, resulting in an unnecessary prolongation of the fault and a considerable release 
of energy. Therefore, the desirable temperature distribution should melt most of the wire. 
For this reason, Guile and Carne?” solved the heat conduction equation 


=a +q(14+ au), —-L<a<L, 0<t, 


to understand the temperature structure within the fuse just before meltdown. The second 
term on the right side of the heat conduction equation gives the resistance heating, which is 
assumed to vary linearly with temperature. If the terminals at x = +L remain at a constant 
temperature, which we can take to be zero, the boundary conditions are u(—L, t) = u(L,t) = 
0,0 <t. The initial condition is u(z,0) = 0, —L < x < L. Find the temperature field as a 
function of the parameters a, q, and a. 


35 Guile, A. E., and E. B. Carne, 1954: An analysis of an analogue solution applied to the heat conduction 
problem in a cartridge fuse. AIEE Trans., Part 1, 72, 861-868. 
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Step 1: Take the Laplace transform of the partial differential equation and boundary con- 
ditions and show that 


@U(z,s)  aq—s 
Age + @ U(x, s) = -737 0<a<L, 
subject to the boundary conditions U(—L, s) = U (L, s) = 0. 


Step 2: Show that the solution to the previous step is 


q q cosh(z ys — aq/a) 


s—aq (s— qaq)cosh(Lys = ag/a) 


sU (x, s) = 


Step 3: Show that U(x,s) has a removable singularity at s = aq and simple poles at 
Sn = aq — (2n — 1)?r?a? /4L? with Vsn — ag = (2n — 1)rai/2L, where n = 1,2,3,.... 


Step 4: Use Bromwich’s integral and show that 


u(x,t) = -4 5 a cos[(2n — 1)rz/2L] explagt — (2n — 1) r?a’t/4L°]. 


Step 5: Integrate u(x,t) with respect to time and obtain 


u(x,t) = K 2 On te — eee {1 — exp[agt — (2n — 1)?1?a7t/4L"]} , 


where the constant of integration ensures that u(z,0) = 0. 


16. Solve?® 
ðu u  20u 


ðt ðr? E r Or’ 
subject to the boundary conditions lim,_,o |u(r, t)| < 00, ur(1,t) = 1, 0 < t, and the initial 
condition u(r,0) =0,0<r<1. 


0<r<1, 0<t, 


Step 1: Introduce the new variable u(r, t) = ru(r,t) and show that the problem becomes 


2 
moe 0<r<1,0<t, 


with the boundary conditions lim,_,9 v(r, t) > 0 and 


PABU a yeas t>0 
Or 


and the initial condition v(r,0) =0,0<r< 1. 


Step 2: Take the Laplace transform of the partial differential equation and boundary con- 
ditions and show that 


dV (r,s) 


72 — sV (r,s) = 0, 0<r<l, 
r 


36 See Reismann, H., 1962: Temperature distribution in a spinning sphere during atmospheric entry. J. 
Aerosp. Sci., 29, 151-159. 
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subject to the boundary conditions lim,_,9 V(r, s) > 0 and V’(1,s) — V(1,s) = 1/s. 
Step 3: Show that the solution to the previous step is 


sinh(r4/s ) 
s [vs cosh(y/s ) — sinh(y/s )] 


V(r,s) = 


Step 4: Show that V(x, s) has a second-order pole at s = 0 and simple poles where ,/s;, = 
iùn, Sn = —À2 and tan(àn) = àn, n = 1,2,3,.... 


Step 5: Use Bromwich’s integral and show that 


3 2M sinQnr) x2; 
Boxe 
10 r <~ A2 sin(An) 


2 
u(r, t) = 5 + 3t 


where tan(An) = An. 


17. Solve?” 
ðu 4 (Pu | 2du\ | a? A (ru) | 
ae (3 + 2%) T q(t) = m Ər? T q(t), b<r<o, 0<t, 


subject to the boundary conditions 


Ou(b, t) 
Or 


t 
= u(b, t), lim u(r, t) = uo +f q(T) dr, 0<t, 
Tr Co 0 


and the initial condition u(r, 0) = uo, b < r < oo. 


Step 1: Take the Laplace transform of the partial differential equation and boundary con- 
ditions and show that 
d{rU(r,s)| s r[Q(s) + uo] 


T3 2 (r,s) =- a b<r <œ, 


subject to the boundary conditions U” (b, s) = U (b, s) and lim,_,.. U(r, s) = [uo + Q(s)]/s. 


Step 2: Show that the solution to the previous step is 


_ uo + Q(s) bQ(s) e7909) buo — e 4("-) 
~~g s(q+1/8) r s(q+1/B) r’ 


U(r, s) 


where q = /s/a and 8 = b/(1 + b). 
Step 3: Because 


T Ce Zoa 


37 See Frisch, H. L, and F. C. Collins, 1952: Diffusional processes in the growth of aerosol particles. J. 
Chem. Phys., 20, 1797—1803. 
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show that 
u(r, t) = uo h — E e] | i i PHF Font | q(t) dr, 
where 
f(r, t) = erfe( 2) exo(" 3 + a) erfe( “0 + z) ; 
18. Consider®® a viscous fluid located between two fixed walls x = +L. At x = 0 we 


introduce a thin, infinitely long rigid barrier of mass m per unit area and let it fall under 
the force of gravity, which points in the direction of positive x. We wish to find the velocity 
of the fluid u(x,t). The fluid is governed by the partial differential equation 


ðu Ou 
— =v 0 L, O0<t 
Ot VAD? <T< L, <t, 


subject to the boundary conditions u(L, t) = 0 and u;(0,t) — 2puz(0,t)/m = g, 0 < t, and 
the initial condition u(x,0) = 0,0 < x < L. 


Step 1: Take the Laplace transform of the partial differential equation and boundary con- 
ditions and show that 


d?U 
CUES) S00, 5) =0, 0<zr< L, 


subject to the boundary conditions U (L, s) = 0 and sU (0, s) — 2uU' (0, s)/m = g/s. 
Step 2: Show that the solution to the previous step is 


UA TE gsinh[(L — x) /s/v] 
s[s sinh(Ly/s/v ) + 2p/s cosh(Ly/s/v )/(my/v )] 


Step 3: Show that U(x, s) has simple poles at s = 0 and sn = — vA? /L?, where An tan(An) = 
2uL/(mv) =k and n = 1,2,3,.... 


Step 4: Use Bromwich’s integral and show that 


mg(L—2) 4gpL? S sin[v\n(L — x)/L] exp(—vA2t/L7) 


t — 
ued) Dy my? 2+ MA +k +4)] Sinn) 


19. Consider? a viscous fluid located between two fixed walls x = +L. At x = 0 we 
introduce a thin, infinitely long rigid barrier of mass m per unit area. The barrier is acted 
upon by an elastic force in such a manner that it would vibrate with a frequency w if the 


38 See Havelock, T. H., 1921: The solution of an integral equation occurring in certain problems of viscous 
fluid motion. Philos. Mag., Ser. 6, 42, 620-628. 


39 See Havelock, T. H., 1921: On the decay of oscillation of a solid body in a viscous fluid. Philos. Mag., 
Ser. 6, 42, 628-634. 
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liquid were absent. We wish to find the barrier’s deviation from equilibrium, y(t). The fluid 
is governed by the partial differential equation 


Ou 3u 
— = V 0 L, O<t. 
dt V2’ <T< L, < 


The boundary conditions are u(L,t) = my” (t) — 2uu,(0,t) + mwy(t) = 0, y(t) = u(0, t), 
0 < t, and the initial conditions are u(z,0) =0,0<a< L, y(0) = A, and y/(0) = 0. 


Step 1: Take the Laplace transform of the partial differential equation and boundary con- 
ditions and show that 


d?U 
CUS) _ Sy(@, 5) =0, 0<a<L, 


subject to the boundary conditions U(L,s) = 0 and ms?Y (s) — 2uU’(0,s) + mw?Y(s) = 
—msA and sY(s) — A = U (0, s). 


Step 2: Show that U(x, s) = Bsinh| /s/o(L — x)|. 
Step 3: Show that at x = 0, 


ms’Y(s) + 24B JŽ cosh jey: + mw’Y (s) = msA 
v v 
sY (s) — A = Bsinh BE ; 
v 


Step 4: Eliminating B in Step 3, show that 


and 


ms + 2uy/s/v coth( £ s/v) 


Y(s)=A ; 
ms? + 2s /s/v coth( L s/v) + mw? 


Step 5: Show that Y (s) has simple poles at An which are the roots of 


A2 + 2yd8/? coth( L dnl) /(m/v) + w? = 0, WH 1,2, Ina 


Step 6: Use Bromwich’s integral and show that 


4u Aw? = Anern? 
y(t) = mL D 4 2u , 2uLyy3 4 2\2 1 6w2p Li y4- 
n=1 An (an) mv A3 + 2w An |! mL An rw 
20. Solve? 
Ou 207u 


— =a > 0 L, O<t 
at Oo? <T< L, <t, 


40 See McCarthy, T. A., and H. J. Goldsmid, 1970: Electro-deposited copper in bismuth telluride. J. 
Phys. D, 3, 697-706. 
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subject to the boundary conditions u,(0,t) = 0, a?u,(L,t) + au(L,t) = F, 0 < t, and the 
initial condition u(x,0) =0,0<a<L. 


Step 1: Take the Laplace transform of the partial differential equation and boundary con- 
ditions and show that 


@U(a,s) s 

peg ag ae), 0<a<L, 
subject to the boundary conditions U’(0,s) = 0 and a?U’(L,s) + aU (L, s) = F/s. 
Step 2: Show that the solution to the previous step is 


F cosh(qx) 


U(x, s) = s[a?qsinh(qL) + acosh(qL)]’ 


where q = \/s/a. 


Step 3: Show that U(zx,s) has simple poles at s = 0 and sn = —a?A2 /L?, where An is the 
nth root of Atan(A) = aL /a?, qn = iAn/L, and n = 1,2,3,.... 


Step 4: Use Bromwich’s integral and show that 


u(x,t) = Eh -nD o E, PAD), 


[RL(1 + AL) + A2) cos(àn) 
where h = a/a?. 


21. Solve 
ðu u 


Ot Ox?’ 
subject to the boundary conditions u(0,t) = 0 and 3a [uz(1,t) — u(1, ¢)] + u(1,t) = 8(t), 
0 < t, and the initial condition u(z,0) = 0,0 < x < 1. Here d(t) denotes the Dirac delta 
function. 


O0<a<l, 0<t, 


Step 1: Take the Laplace transform of the partial differential equation and boundary con- 
ditions and show that 


dU(z, 8) 
dx? 


subject to the boundary conditions U(0,s) = 0 and 3a[U’(1, s) — U(1,s)] + sU(1,s) =1. 


— sU(x,s) =0, 0<2<1l, 


Step 2: Show that the solution to the previous step is 


Cn sinh(2¥/s ) 
i 3a [Vs cosh( vs ) — sinh (vs )] + s sinh( vs ) 


Step 3: Show that U(x,s) has simple poles at s = 0 and sn = —A? where An cot(An) = 
(3a + à2)/3a, n = 1,2,3,.... 
Step 4: Use Bromwich’s integral and show that 


2 


n=1 


sin(Anz) exp(—A?t) 


u(a,t) [3a +3 + A2/(3a)] sin(An) ` 


aes | 
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22. Solve*! the partial differential equation 


Ou Ou 3u 
Ot Ox Ox?’ 


O0<a<l1, 0<t, 
where V is a constant, subject to the boundary conditions u(0,t) = 1, uz(1,t) = 0, 0 < t, 
and the initial condition u(#,0) =0,0<a<1. 


Step 1: Take the Laplace transform of the partial differential equation and boundary con- 
ditions and show that 


2 
EUG) yee) sU(z,s) =0, 0<a<1, 

dx? dx 
subject to the boundary conditions U(0,s) = 1/s and U’(1,s) = 0. 


Step 2: Show that the solution to the previous step is 


eV2/2 weosh|u(1 — x)] + (V/2) sinh[u(1 — 2)| 
s [u cosh(u) + (V/2) sinh(1.)] 
adv cosh (1 = x)Vs| +(V/2) sinh| (1 _2)/s | 


(s! — V2/4) | Vs" cosh ( v3 ) + (V/2) sinh( v5" )| | 


U(a,s) = 


=e 


where u = \/s + V2/4 and s’ = s + V? /4. 
Step 3: Show that U(x, s) has simple poles at s’ = V?/4 and s’, = —A? with An cot(ÀAn) = 
—V/2, where n = 1,2,3,.... 


Step 4: Using Bromwich’s integral, show that 


u(a,t) =1— 2eV2/2-V°t/4 


Co 


S An{(V/2) sin[An (1 — £)] + An cos[An (1 — x)]}e >t 
(A2 + V2/4)[An sin(An) — (1 + V/2) cos(An)] ` 


n=1 


23. Solve’? the partial differential equation 


Oru Ou Ou 
dzat | Ot + b—— = 0, 0<r<w, 0<a,b,t, 


subject to the boundary conditions u(0,t) = 1, limz-... u(x,t) > 0, 0 < t, and the initial 
condition uz(x#,0) + au(xz,0) =0,0<4<o. 


Step 1: Take the Laplace transform of the partial differential equation and 


(s +b)U' (x, s) + asU(a, s) = 0, 0< z<, 


41 See Yoo, H., and E.-T. Pak, 1996: Analytical solutions to a one-dimensional finite-domain model for 
stratified thermal storage tanks. Sol. Energy, 56, 315-322. 


42 See Liaw, C. H., J. S. P. Wang, R. A. Greenhorn, and K. C. Chao, 1979: Kinetics of fixed-bed 
absorption: A new solution. AICHE J., 25, 376-381. 
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subject to the boundary conditions lim,_,9 |U (x, s)| < œœ and U(0,s) =1/s. 
Step 2: Show that the solution to the previous step is 


1 asx 
U(a,s) = sexp(- =) ; 


Step 3: Because 


show that 


Step 4: By inverting U(x, s) term by term and using the first shifting theorem, 


u(x,t) =1— a e "Ip (2 bin ) dn. 
0 


24. Solve 


13 / Ou ðu 
= £ < 
a (rs) a d(t), O<r<a, 0<t, 


subject to the boundary conditions lim,_,o |u(r, t)| < oo, u(a, t) = 0, 0 < t, and the initial 
condition u(r,0) = 0, 0 < r < a, where ô(t) is the Dirac delta function. Note that 
Jn (iz) = i"I,(z) and I,(iz) = i” J,(z) for all complex z. 


Step 1: Take the Laplace transform of the partial differential equation and boundary con- 
ditions and show that 
dU (r,s) 
r 
r dr dr 


sU(r,s) = 1, 0<r<a, 


subject to the boundary conditions lim,_,9 |U (r, s)| < oo and U (a, s) = 0. 
Step 2: Show that the solution to the previous step is 


— Lolrvs) — Jolav’s) 
s Ip(av/s) 


Step 3: Show that U(r,s) has a removable singularity at s = 0 and simple poles at sn = 
—k? /a*, where Jo(kn) = 0 and n = 1,2,3,.... 


Step 4: Using Bromwich’s integral, show that 


Jo(k ae echt /a? 
u(r, t) =-2)° bath n : 


25. Solve 
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subject to the boundary conditions lim,_,9 |u(r, t)| < co, u(a,t) = 0, 0 < t, and the initial 
condition u(r,0) = 0,0<r<a. Note that J, (iz) = i"I,(z) and I, (iz) = i” J,(z) for all 
complex z. 
Step 1: Take the Laplace transform of the partial differential equation and boundary con- 
ditions and show that 

„U (r,s) 
rdr dr 


1 
sU (r,s) =—-, 0<r<a, 
s 


subject to the boundary conditions lim,_,9 |U (r, s)| < oo and U (a, s) = 0. 
Step 2: Show that the solution to the previous step is 
In(ay) — nr) 

s lays) 


Step 3: Show that U(r, s) has simple poles at s = 0 and sn = —k2/a? where Jo(kn) = 0 
and n = 1,2,3,.... 


U(r,s) = 


Step 4: Using Bromwich’s integral, show that 


u(r, t) = 


Jo(k te 2472 
a2 o( en knt/a 
eet k3 Ti (kn) Jı(k ` 


26. Solve 


AE Garaa Ot 
ðt rôr "Or ; ie ; 


subject to the boundary conditions lim,—49 |u(r,t)| < 00, u(a,t) = e~/, 0 < t, and the 
initial condition u(r,0) = 1,0 <r <a. Note that J, (iz) = i” In(z) and I,(iz) = i"Jn(z) 
for all complex z. 
Step 1: Take the Laplace transform of the partial differential equation and boundary con- 
ditions and show that 

dU (r,s) 

i 

r dr dr 


sU(r,s) = —1, O0<r<a, 


subject to the boundary conditions lim,_,9 |U (r, s)| < oo and U(a, s) = 1/(s + 1/79). 
Step 2: Show that the solution to the previous step is 


i. 1 DEANE 
E (= Po 


Step 3: Show that U (r,s) has simple poles at s = 0, s = —1/79, and sn = —k?/a?, where 
Jo(kn) = 0 and n = 1,2,3,.... 


Step 4: Using Bromwich’s integral, show that 


Jar al b 
ioe an rm e—t/To 4. 2a? 2 = knr/a) okt /a? 
0 


eas J1 (Kn ie 


— eo t/to 242 Jo(knr/a) —k2t/a? _ „—t/To 
PEE IAR Ba (e et) 
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27. Solve 


Ou o/u 16u 
—— >= mA < 
Ot (Sate) TSS es Uet 


subject to the boundary conditions 


lim ju(r,t)| <œ, u(b,t) = kt, 0<t, 
and the initial condition u(r,0) = 0,0 <r <b. Note that J,(iz) = i” In(z) and I, (iz) = 
i” J,(z) for all complex z. 


Step 1: Take the Laplace transform of the partial differential equation and boundary con- 
ditions and show that 


ld|{d 
b ne] U(r, s) = 0, 0<r<b, 


r dr dr a? 


subject to the boundary conditions lim,_,9 |U (r, s)| < oo and U (b, s) = k/s?. 
Step 2: Show that the solution to the previous step is 


k Io(r/s/a) 


Ug Tet fala 


Step 3: Show that U(r, s) has a second-order pole at z = 0 and simple poles at ikn = \/Zn/a 
Or Zn = —a7K2, where Jo(k,b) = 0 and n = 1,2,3,.... 


Step 4: Using Bromwich’s integral, show that 


b2 — oS Jo( Kn? 
u(r,t) =k l na H 2b ape oilt) >i 


28. Solve the nonhomogeneous heat equation for the spherical shell? 


ðu 3 (3 20u A 


a Ti) a<rx<ß, 0<t, 


subject to the boundary conditions u,(a,t) = u(G,t) = 0, 0 < t, and the initial condition 
u(r,0) =O,a<r< B. 


Step 1: By introducing v(r,t) = ru(r,t), show that the problem simplifies to 


ðv (v A 
pa (Sa+5). a<r< 8, 0<t, 


subject to the boundary conditions v,(a,t) — v(a, t)/a = v(6,t) = 0, 0 < t, and the initial 
condition u(r,0) =O,a<r< B. 


43 See Malkovich, R. Sh., 1977: Heating of a spherical shell by a radial current. Sov. Phys. Tech. Phys., 
22, 636. 
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Step 2: Taking the Laplace transform of the differential equation and boundary conditions 
in Step 1, show that 


A 
V8) = -—;, a<r< Bp, 
a sr 


along with V’ (a, s) + V(a,s)/a = V(8,s) =0. 


Step 3: Using the method of variation of parameters, show that the particular solution to 
Step 2 is Vp(r, s) = u1 (r, s) cosh(gr) + ua(r, s) sinh(gr), where 


u(r, 8) = “| LCa) dr, ua(r, s) = i ka) dr, and q= vs/a. 
B B 


sq 73 sq 73 


Step 4: Show that the general solution to Step 2 is 


V(r, s) = Csinh[q(r — 8)] à a sinh[g(r = 7)] yp. 


This solution satisfies V (8, s) = 0. 
Step 5: Use the remaining boundary condition and show that 


A { sinh[q(8 — r)] | “aq cosh(qn) + sinh(qn) d f P=" sinh(qn) \ 
n dn ¢, 
0 ) 


U — 
(r, 8) srq | aq cosh(gé) + sinh(q£) (a+)3 (r+) 


where £ = 8—a. Note: V(r,s) =rU(r, 8). 


Step 6: Show that U(r,s) has simple poles at s = 0 and sn = —a?y2, where yn is the nth 
root of ayn, cos(Yné) + sin(ynl) = 0, n = 1,2,3,.... 


Step 7: Use Bromwich’s integral and show that 


aoa a): 190) 


B —r)lexp(—a?72t) f1 sin(yn£n) 
A | C-n an}, 


2a? & sin[yn 


rl? sin? 
n=0 


where Yn is the nth root of ay + tan(¢y) = 0, and ô = 1 + a/£. 


2.6 THE SOLUTION OF LAPLACE’S EQUATION BY LAPLACE TRANSFORMS 


Laplace transforms are useful in solving Laplace’s or Poisson’s equation over a semi- 
infinite strip. The following problem illustrates this technique. 
Let us solve Poisson’s equation within a semi-infinite circular cylinder 


10 / Ou u 2 
Or (3) + ae zrel — b), O<r<a, 0<z<~aM, (2.6.1) 


subject to the boundary conditions 


u(r, 0) = 0, lim lu(r,z)|< oo, O<r<a, (2.6.2) 
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and 
u(a, z) =0, 0<z<~m, (2.6.3) 


where 0 < b < a. This problem gives the electrostatic potential within a semi-infinite cylin- 
der of radius a that is grounded and has the charge density of n(z) within an infinitesimally 
thin shell located at r = b. 

Because the domain is semi-infinite in the z direction, we introduce the Laplace trans- 
form 


U(r, s) =| u(r, z)e °* dz. (2.6.4) 
0 
Thus, taking the Laplace transform of Equation 2.6.1, we have that 
dU(r,s)] | 9 _ 2 
= aie za Tr + s“U (r,s) — su(r,0) — uz(r,0) = zrol — b). (2.6.5) 


Although u(r,0) = 0, uz(r,0) is unknown and we denote its value by f(r). Therefore, 
Equation 2.6.5 becomes 


ea] t s?U(r,s) = f(r) + ÉN (s)ô(r =b), O<r<a, (2.6.6) 


rdr i dr 
with lim,_,9 |U (r, s)| < co, and U (a, s) = 0. 
To solve Equation 2.6.6 we first assume that we can rewrite f(r) as the Fourier-Bessel 
series 


r) = X. AnJo(knr/a), (2.6.7) 


n=1 


where kn is the nth root of the Jo(k) = 0, and 


= of f(r) Jo(knr/a) r dr. (2.6.8) 


” aI a? J? (kn) 
Similarly, the expansion for the delta function is 


2b X Jo(knb/a) Jo(kn co), 


ô b) = 2.6.9 
= Be) (2.6.9) 
because r 
| d(r — b)Jo(knr/a)r dr = b Jo(knb/a). (2.6.10) 
0 
Why we chose this particular expansion will become apparent shortly. 
Thus, Equation 2.6.6 may be rewritten as 
ld| dU(r,s)| | 2N(s A A a 
a b A H s°U(r, s) = 2 i 5 Jo(knr/a), (2.6.11) 


where a, = AFl r) Jo(knr/a)r dr. 
The form of the right side of Equation 2.6.11 suggests that we seek solutions of the 
form 


s) = BnJolknr/a), O<r<a. (2.6.12) 


n=1 
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We now understand why we rewrote the right side of Equation 2.6.6 as a Fourier-Bessel 
series; the solution U(r, s) automatically satisfies the boundary condition U (a, s) = 0. Sub- 
stituting Equation 2.6.12 into Equation 2.6.11, we find that 


ee ge (s2 - TE Th o(knr/a), O<r<a. (2.6.13) 


We have not yet determined ag. Note, however, that in order for the inverse of Equation 
2.6.13 not to grow as e¥»?/4, the numerator must vanish when s = kp /a and s = kn/a is a 
removable pole. Thus, 

ak = —2N (kn /a)Jo(knb/a), (2.6.14) 


and 


=e ee nae Jo(knr/a), OSr<a. (2.6.15) 


r,s 


The inverse of U(r, s) then follows directly from simple inversions, the convolution theorem, 
and the definition of the Laplace transform. The complete solution is 


u(r, z) = ° y Jo(kinb/a)Jo(knr/a) 


a kn J2(Kn) 
x / n(r)ekn@—T/a dr- | TO dr 
0 0 
= J n(r)e kn 7/4 ekn2/4 dr + I Mae eens ar| (2.6.16) 
0 0 
_2 3 Jo(kinb/a) Jo(knr/a) (2.6.17) 
a kn J2 (Kn) 


x ff nfrje sre dr | nije ele dr — f net dr, 
0 0 z 


Problems 
1. Use Laplace transforms to solve 
Ou du 
Be op 0<zr<œ, 0<y<a, 


subject to the boundary conditions u(0,y) = 1, lims lulz, y)| < œ, 0 < y < a, and 
u(x, 0) = u(x,a)= 0,0 < £z < œ. 


Step 1: Take the Laplace transform of the partial differential equation and boundary con- 


ditions and show that 2 
U 
age +377 =s + f(s,y), 


subject to the boundary conditions U(s,0) = U(s,a) = 0. 
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Step 2: Because 


_2 = [1 —(- . (nny 
a D'I sin ) 
aoe a 
and expanding f(s, y) in a half-range sine expansion: 


)= Yo An sin (2T ”), where An = f f(s, y )sin( at) dy, 


show that the differential equations in Step 1 can be rewritten 
à U 2 2s[1 — (—1)”] . [NTY 
Z- 4 8U = An (=) , 
z +8 2 a an sin F 
Step 3: Show that the solution of the differential equation in Step 2 is 


Z YPY] +nrAn . (nry 
U(s,y) D aa 27a) sin( a E 


Step 4: For the solution to remain finite as z — oo, s = n/a cannot be a pole of the 
transform U(s,y). Show that A, = —2[1 — (—1)"]/a and 


a 


sie : 5 = e| ee sin] OM a l 


2. Use Laplace transforms to solve 


2 
(rot) + Tio, O<r<a, 0<2z< œ, 


rOr\ Or 


subject to the boundary conditions u(r,0) = 1, lim,_,. |u(r,z)| < œ 0 < r < a, and 
lim,—s0 |u(r, z)| < co, u(a, z) = 0, 0 < z < ov. 


Step 1: Take the Laplace transform of the partial differential equation and boundary con- 
ditions and show that 


ld | a 


H3? = < 
gE T sU(r,s)=st+ f(r) O<r<a, 


with |U (0, s)| < oo and U(a,s) = 0. 
Step 2: Rewrite f(r) as the Fourier-Bessel series: 


r)= 5 AnJo(knr/a), 


n=1 
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where kpn is the nth root of the Jo(k) = 0 and 


An = 5 f to ) Jo(Knr/a) r dr. 


ar a? J? (kn) 
Step 3: Because 


Jo(knr/a) 
=2 < 
Se O0<r<a, 


show that the differential equation in Step 1 becomes 
1 d [| dU(r,s) sa? Jı (kn) — knak knr 
H r,s) = J 3 
rdr b dr 5 a? 2 kn pa n) oa 


where ap = fọ F(r)Jolknr/a)r dr. 


Step 4: Show that the solution to the differential equation is 


sa 2 Ti(k n) — knak 
Ca a? 2 an Vin J2 (Kn) Jo(knr/a), O<r<a. 


Step 5: Because s = k,,/a cannot be a pole of U(r, s), ax = aJı (kn). Therefore, 


Step 6: Find the inverse of U (r,s) and show that 


Jo(k ear oo knz/a 
(722 yy eee a ith : 


Further Readings 
Debnath, L., and D. Bhatta, 2015: Integral Transforms and Their Applications. CRC 
Press, 792 pp. A book that covers Laplace, Fourier, z-, Hankel, Mellin, Hilbert and Stieltjes 


transforms and their application. 


Duffy, D. G., 2015: Transform Methods for Solving Partial Differential Equations. CRC 
Press, 728 pp. This book covers the material of this chapter in greater depth. 


Simpson’s Simpson’s 
3/8-rule 1/3-rule 


Trapezoidal 
Rule 


Ratio of quadrature amplitudes to ideal integration 


Chapter 3 
The Z-Transform 


Since the Second World War, the rise of digital technology has resulted in a corre- 
sponding demand for designing and understanding discrete-time (data sampled) systems. 
These systems are governed by difference equations in which members of the sequence yn 
are coupled to each other. 

One source of difference equations is the numerical evaluation of integrals on a digital 
computer. Because we can only have values at discrete time points tg = kT for k = 
0,1,2,..., the value of the integral y(t) = I f(r) dr is 


kT (k-1)T kT 
y(kT) = i (7) ar= [ f(r) tf, f(r) dr (3.0.1) 
kT 
=yk-DT]+ fo foyer = ye 1T] + 77D), (3.0.2) 
(k-1)T 


because ae f(r) dr = Tf(kT). The right side of Equation 3.0.2 is an example of a 
first-order difference equation because the numerical scheme couples the sequence value 
y(kT) directly to the previous sequence value y[|(k — 1)T]. If Equation 3.0.2 had contained 
y|(k — 2)T], then it would have been a second-order difference equation, and so forth. 

Although we could use the conventional Laplace transform to solve these difference 
equations, the use of z-transforms can greatly facilitate the analysis, especially when we 
only desire responses at the sampling instants. Often the entire analysis can be done using 
only the transforms and the analyst does not actually find the sequence y(kT). 

In this chapter we will first define the z-transform and discuss its properties. Then we 
will show how to find its inverse. Finally, we shall use them to solve difference equations. 
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Figure 3.1.1: Schematic of how a continuous function f(t) is sampled by a narrow-width pulse sampler 
f(t) and an ideal sampler fs(t). 


3.1 THE RELATIONSHIP OF THE Z-TRANSFORM TO THE LAPLACE TRANSFORM! 


Let f(t) be a continuous function that an instrument samples every T units of time. 
We denote this data-sampled function by f§(t). See Figure 3.1.1. Taking €, the duration of 
an individual sampling event, to be small, we may approximate the narrow-width pulse in 
Figure 3.1.1 by flat-topped pulses. Then f%(t) approximately equals 


HOR : > f(nT) [H(t — nT + €/2) — H(t -nT — €/2)], (3.1.1) 
n=0 


ife<T. 

Clearly the presence of € is troublesome in Equation 3.1.1; it adds one more parameter 
to our problem. For this reason we introduce the concept of the ideal sampler, where the 
sampling time becomes infinitesimally small so that 


= So f(nT)6(t — nT). 


n=0 


fs(t) = lim $ f(nT) 


n=0 


Ae SE) 
(3.1.2) 


Let us now find the Laplace transform of this data-sampled function. From the linearity 
property of Laplace transforms, 


Fs(s) = Li fs(t)] =£ | X fT)ólt- nT) 


n=0 


= > f(nT)L[6(t — nT]. (3.1.3) 
n=0 


1 Gera (Gera, A. E., 1999: The relationship between the z-transform and the discrete-time Fourier 
transform. IEEE Trans. Auto. Control, AC-44, 370-371) explored the general relationship between 
the one-sided discrete-time Fourier transform and the one-sided z-transform. See also Naumovié, M. B., 
2001: Interrelationship between the one-sided discrete-time Fourier transform and one-sided delta transform. 
Electr. Engng., 83, 99-101. 
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Because £L[5(t — nT)| = e~"*", Equation 3.1.3 simplifies to 
Fs(s) = X fT) T. (3.1.4) 
n=0 


If we now make the substitution that z = e°?, then F'g(s) becomes 


F(2) = Z( fa) = X dee (3.1.5) 
n=0 


where F(z) is the one-sided z-transform? of the sequence f(nT), which we shall now denote 
by fn. Here Z denotes the operation of taking the z-transform while Z~! represents the 
inverse z-transformation. We will consider methods for finding the inverse z-transform in 
Section 3.3. 

Just as the Laplace transform was defined by an integration in t, the z-transform is 
defined by a power series (Laurent series) in z. Consequently, every z-transform has a region 
of convergence that must be implicitly understood if not explicitly stated. Furthermore, 
just as the Laplace integral diverged for certain functions, there are sequences where the 
associated power series diverges and its z-transform does not exist. 

Consider now the following examples of how to find the z-transform. 


e Example 3.1.1 


Given the unit sequence fn = 1, n > 0, let us find F(z). Substituting fn into the 
definition of the z-transform leads to 


F(z) =o 2% =— (3.1.6) 


n=0 


because Xpo 27” is a complex-valued geometric series with common ratio z7}. This series 
converges if |z~+| < 1 or |z| > 1, which gives the region of convergence of F(z). 

MATLAB’s symbolic toolbox provides an alternative to the hand computation of the 
z-transform. In the present case, the command 
>> syms z; syms n positive 
>> ztrans(1,n,z) 
yields 


ans = 


z/(z-1) 


e Example 3.1.2 


Let us find the z-transform of the sequence 


faze, n>0, (3.1.7) 


2 The standard reference is Jury, E. I., 1964: Theory and Application of the z-Transform Method. John 
Wiley & Sons, 330 pp. 
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for a real and a imaginary. 
For a real, substitution of the sequence into the definition of the z-transform yields 


F(z) = 5 eg Dh Car ae (3.1.8) 
n=0 n=0 


If u =e“ z—!, then Equation 3.1.8 is a geometric series so that 


F(z)=ġ u= S (3.1.9) 
n=0 


l-—wu 
Because |u| = e~°7 |z~1], the condition for convergence is that |z| > e~¢7. Thus, 


F(z) = So |z| > e77. (3.1.10) 
AE 


For imaginary a, the infinite series in Equation 3.1.8 converges if |z| > 1, because 
|u| = |z—1| when a is imaginary. Thus, 
z 
F(z) = T= aT? |z| >1. (3.1.11) 
Although the z-transforms in Equation 3.1.10 and Equation 3.1.11 are the same in these 
two cases, the corresponding regions of convergence are different. If a is a complex number, 
then 
z 


F(z) = Sear |z| > eres (3.1.12) 


Checking our work using MATLAB, we type the commands: 
>> syms a z; syms n T positive 


>> ztrans(exp(-a*n*T),n,Zz); 
>> simplify (ans) 


which yields 
ans = 
zxexp(a*T)/(zxexp(a*T)-1) 


e Example 3.1.3 
Let us find the z-transform of the sinusoidal sequence 
fn = cos(nwT), n> 0. (3.1.13) 
Substituting Equation 3.1.13 into the definition of the z-transform results in 


F(z) = 3 cos(nwT)z™”. (3.1.14) 


From Euler’s formula, 
cos(nwT) = 4 (eT + e—imeT), (3.1.15) 
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so that Equation 3.1.14 becomes 


1 /,; 
F(z) = 5 5 Cae 4 ee) 
n=0 
or 
F(z) = see) oe Zed) 
From Equation 3.1.11, 
Berner) me = |z| Soa 
z—et 


Substituting Equation 3.1.18 into Equation 3.1.17 and simplifying yields 


z|z — cos(wT)| 


F(z) = 1. 
(2) 22 — 2z cos(wT) +1’ ele 
e Example 3.1.4 
Let us find the z-transform for the sequence 
7 1, O<n<5, 
In=9 GP, 6<n. 
From the definition of the z-transform, 
5 foe) 1 n 
zizro yS (}) 
n=0 n=6 
eee rr oar P 1 1 "i 2z 
E z z2 g8 z4 2 2-1 
1 1 1 1 1 1 


2z 422 823 1624 3225 
2z 1 Bie i 7 15 31 


ee 422 ` 


823 i 1624 ja 3225 
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(3.1.16) 


(3.1.17) 


(3.1.18) 


(3.1.19) 


(3.1.20) 


(3.1.21) 


(3.1.22) 


(3.1.23) 


We could also have obtained Equation 3.1.23 via MATLAB by typing the commands: 


>> syms Z; syms n positive 
>> ztrans(?’1+((1/2)^`n-1)»*Heaviside(n-6)’,n,z) 


which yields 


ans = 
2*z/(2*z-1)+1/2/z+3/4/z^2+7/8/z^3+15/16/z^4+31/32/z^5 
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We summarize some of the more commonly encountered sequences and their transforms 
in Table 3.1.1 along with their regions of convergence. 


e Example 3.1.5 


In many engineering studies, the analysis is done entirely using transforms without 
actually finding any inverses. Consequently, it is useful to compare and contrast how various 
transforms behave in very simple test problems. 

Consider the time function f(t) = ae~“ H(t), a > 0. Its Laplace and Fourier transform 
are identical, namely a/(a + iw), if we set s = iw. In Figure 3.1.2 we illustrate its behavior 
as a function of positive w. 

Let us now generate the sequence of observations that we would measure if we sampled 
f(t) every T units of time apart: fa = ae~*"". Taking the z-transform of this sequence, it 
equals az/ (z — ewe) Recalling that z = eT = eT, we can also plot this transform as 
a function of positive w. For small w, the transforms agree, but as w becomes larger they 
diverge markedly. Why does this occur? 

Recall that the z-transform is computed from a sequence comprised of samples from a 
continuous signal. One very important flaw in sampled data is the possible misrepresenta- 
tion of high-frequency effects as lower-frequency phenomena. It is this aliasing or folding 
effect that we are observing here. Consequently, the z-transform of a sampled record can 
differ markedly from the corresponding Laplace or Fourier transforms of the continuous 
record at frequencies above one half of the sampling frequency. This also suggests that care 
should be exercised in interpolating between sampling instants. Indeed, in those applica- 
tions where the output between sampling instants is very important, such as in a hybrid 
mixture of digital and analog systems, we must apply the so-called “modified z-transform.” 


Problems 


From the fundamental definition of the z-transform, find the transform of the following 
sequences, where n > 0. Then check your answer using MATLAB. 


IA” ino 2 I O0<n<5, 
1 n= (5) Co mt Ley 5<n 
e a n=0,1,...,10 F {= aN 
-Jn 5 n eJa 3574, n= 4, 

(i ’ nzil a”, n>2 


3.2 SOME USEFUL PROPERTIES 


In principle we could construct any desired transform from the definition of the z- 
transform. However, there are several general theorems that are much more effective in 
finding new transforms. 
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Table 3.1.1: Z-Transforms of Some Commonly Used Sequences 


17. 


18. 


fh, n>0 


fo = k = const. 
fn = 9, nol 


fm = k = const. 


fn = 9, for all values 
ofn #m 


k = constant 
kn 
kn? 


ke~*"T, a complex 


—anT 


kne , a complex 


sin(wonT) 
cos(wonT ) 
eT sin(wonT) 
—anT 


e cos(wonT) 


n 


a” , a constant 


sinh(wonT’) 
cosh(wonT’) 
a” /n! 


[In(a)]"/n! 


F(z) 


kee 


kz/(z— 1) 
kz/(z—1)? 
kz(z+1)/(z-1)8 


kz/ (z—e-47) 


kze~?? 
G@—e-4T)2 


zsin(woT) 
z2—2z cos(woT)+1 


z[z—cos(woT)] 
z2—2z cos(woT)+1 


ze tT sin(woT) 


z2—2ze—°T cos(woT)+e7 247 


ze tT [ze*? —cos(woT)| 


z2—2ze—°T cos(woT)+e7 247 


z/(z—a@) 
az/(z—«a)? 


az(z + a)/(z — a)’ 


zsinh(woT) 
2z2—2z cosh(woT)+1 


z[z—cosh(woT)] 
z?—2z cosh(woT)+1 


Region of 
convergence 


|z| > 0 


z| >0 


k| > 1 

|z] > 1 

k| > 1 
|z| > [e777 | 
|z| > |e? | 
z|>1 


z|>1 


|z| >e 


|z| >e 
lel > lal 
lel > lal 
lel > lal 
|z| > cosh(woT) 
|z| > sinh(woT) 
|z| >0 


|z| >0 
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amplitude of transform 
3 


0.0 2.5 5.0 7.5 10.0 12.5 
(A) 


Figure 3.1.2: The amplitude of the Laplace or Fourier transform (solid line) for the function ae™ ®t H(t) 
and the z-transform (dashed line) for the sequence fn = ae™®”T as a function of frequency w for various 
positive values of a and T = 1. 


Linearity 


From the definition of the z-transform, it immediately follows that 
if hn =cCifpatecegn, then H(z) =a F(z) +ceG(z), (3.2.1) 


where F(z) = Z(fn), G(z) = Z(gn), H(z) = Z(hn), and c1, c2 are arbitrary constants. 


Multiplication by an expo- 
nential sequence 


If ga =e Tfn n>0, then G(z) = F(ze). (3.2.2) 


This follows from 
G(z) = Z(9n) = Xo Inz” = 5 eg. = 5 fies) = F(ze°T). (3.2.3) 
n=0 n=0 n=0 


This is the z-transform analog to the first shifting theorem in Laplace transforms. 


Shifting 


The effect of shifting depends upon whether it is to the right or to the left, as Table 
3.2.1 illustrates. For the sequence f,—2, no values from the sequence fn are lost; thus, 
we anticipate that the z-transform of f,—-2 only involves F(z). However, in forming the 
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Table 3.2.1: Examples of Shifting Involving Sequences 


n Fn Jn-2 frn+e2 
0 1 0 4 
1 2 0 8 
2 4 1 16 
3 8 2 64 
4 16 4 


sequence fn+2, the first two values of fn are lost, and we anticipate that the z-transform of 
fn+2 cannot be expressed solely in terms of F(z) but must include those two lost pieces of 
information. 

Let us now confirm these conjectures by finding the z-transform of fn41, which is a 
sequence that has been shifted one step to the left. From the definition of the z-transform, 
it follows that 


2(fn+i) = > fany ” =z bD fue (3.2.4) 
n=0 
or 
Z2(fasi) = she + zfo— zfo, (3.2.5) 


where we added zero in Equation 3.2.5. This algebraic trick allows us to collapse the first 
two terms on the right side of Equation 3.2.5 into one and 


Z(fn+1) = 2F(z) — z fo. (3.2.6) 
In a similar manner, repeated applications of Equation 3.2.6 yield 
Sige) = 2 ee ig a fy a mei (3.2.7) 


where m > 0. This shifting operation transforms fn+m into an algebraic expression involv- 
ing m. Furthermore, we introduced initial sequence values, just as we introduced initial 
conditions when we took the Laplace transform of the nth derivative of f(t). We will make 
frequent use of this property in solving difference equations in Section 3.4. 

Consider now shifting to the right by the positive integer k, 


gn = fn—-kHn-k, n2 0, (3.2.8) 
where H,_, = 0 for n < k and 1 for n > k. Then the z-transform of Equation 3.2.8 is 
G(z) =z * F(z), (3.2.9) 


where G(z) = Z(g,), and F(z) = Z(f,,). This follows from 


G(2) =X gaz" =X Hehe =z 2 far Ok = fmz ™™ = zF F(z). 
n=0 n=0 


(3.2.10) 
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This result is the z-transform analog to the second shifting theorem in Laplace transforms. 


In symbolic calculations involving MATLAB, the operator H,_, can be expressed by 
Heaviside(n-k). 


| Initial-value theorem 


The initial value of the sequence fn, fo, can be computed from F(z) using the initial- 
value theorem: 


fo = lim F(z). (3.2.11) 


From the definition of the z-transform, 
F(z) = Xo he = fot fiz? + fha tE e (3.2.12) 
n=0 
In the limit of z —> œ, we obtain the desired result. 


| Final-value theorem 


The value of fn, as n > œ, is given by the final-value theorem: 


foo = lim (z — 1)F(z), (3.2.13) 
z>1 
where F(z) is the z-transform of fn. 


We begin by noting that 


n 


Z(fnsi— fn) = lim X feta — fe). (3.2.14) 


k=0 
Using the shifting theorem on the left side of Equation 3.2.14, 


n 


zF(2) = zfo — F(z) = lim Nra — fae. (3.2.15) 
k=0 


Applying the limit as z approaches 1 to both sides of Equation 3.2.15: 


n 


lim (2—1)F(z) — fo = lim X (fesi — fe) (3.2.16) 
k=0 
= lim [(f1 — fo) + (fo — fa) +--+ + (fn — fn) + Fn — fn) +] 
(3.2.17) 
= Jim (—fo + fn+1) = — fo + fo- (3.2.18) 
Consequently, 


fæ = lim (z= 1)F (2). (3.2.19) 
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Note that this limit has meaning only if fə exists. This occurs if F(z) has no second-order 
or higher poles on the unit circle and no poles outside the unit circle. 


| Multiplication by n 


Given 
In =Nfn, n> 0, (3.2.20) 
this theorem states that P? 
aona (3.2.21) 
dz 


where G(z) = Z(gn), and F(z) = Z( fn). 


This follows from 


nn 2 a ro dF(z) 
= Ro, Tran n—-1 _ 
G(z) = Don? = Doin =a nfaz =o S (3.2.22) 


n=0 
| Periodic sequence theorem 


Consider the N-periodic sequence: 


fn ={fofife...fw—1 fofi), (3.2.23) 
eS 
first period 
and the related sequence: 
_ J fas O<n<N-I, 
t= { 0, We Hi, (3.2.24) 


This theorem allows us to find the z-transform of fn if we can find the z-transform of £n 
via the relationship 


X 
F(z) = o |> 1, (3.2.25) 
1—2-N 
where X(z) = Z(an). 
This follows from 
oo N-1 2N-1 3N-1 
F(z) = X fax = So ane Pt So anon” So nonz +++. (3.2.26) 
n=0 n=0 n=N n=2N 


Application of the shifting theorem in Equation 3.2.26 leads to 


F(z) =X(z) +2 NX(z) 420° X(2)+ =X(z) [L420 A e]. (8.2.27) 
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Equation 3.2.27 contains an infinite geometric series with common ratio z7, which con- 
verges if |z7^]| <1. Thus, 


| >1. (3.2.28) 


| Convolution 


Given the sequences fn and gn, the convolution product of these two sequences is 


n n 
wh = fat gu > eck = Y Jakik: (3.2.29) 
k=0 k=0 


Given F(z) and G(z), we then have that W (z) = F(z)G(z). 
This follows from 
were [Ermi Ene ez 
n=0 Lk=0 n=0 k=0 
because gn—p~ = 0 for k > n. Reversing the order of summation and letting m = n — k, 


MES YS ua = [S n [Ze 
k=0 m=0 


k=0 m=—k 


=F(z)G(z). (3.2.31) 


We can use MATLAB’s command conv( ), which multiplies two polynomials to perform 
discrete convolution as follows: 


>x = [1111111]; 
>>y = [1 2 4 8 16 32 64]; 
>>z = conv(x,y) 

produces 

z= 


1 3 7 15 31 63 127 126 124 120 112 96 64 


The first seven values of z contain the convolution of the sequence x with the sequence y. 
Consider now the following examples of the properties discussed in this section. 


e Example 3.2.1 


From 1 
Z(a") = 3.2.32 
a=, (3.2.32) 
for n > 0 and |z| > |a| , we have that 
f 1 
Z(e"*) (3.2.33) 
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and í 
Bie) = 


~ [= eitz 1? 


if n > 0 and |z| > 1. Therefore, the sequence fn = cos(nx) has the z-transform 


F(z) = Z{cos(n2)| = z(e”) +Æ iz(e=i"2) 
1 1 ae 1 = 1 — cos(x)z 7! 


~ 21—e@z-1 " 21—e-#z-1  1—2cos(e)z 1 +z? 


e Example 3.2.2 


Using the z-transform, 


we find that 


e Example 3.2.3 
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(3.2.34) 


(3.2.35) 


(3.2.36) 


(3.2.37) 


(3.2.38) 


(3.2.39) 


Consider F(z) = 2az~1/(1 — az~')3, where |a| < |z| and |a| < 1. Here we have that 


—1 


fo = lim F(z) = lim ——\ = 0 


z= 00 Zoo (1 — az—1)8 
from the initial-value theorem. This agrees with the inverse of F(z): 


fn =n(n4+ la”, n> 0. 


(3.2.40) 


(3.2.41) 


If the z-transform consists of the ratio of two polynomials, we can use MATLAB to find 


fo. For example, if F(z) = 227/(z — 1)’, we can find fo as follows: 


>>num = [2 0 0]; 
>>den = conv([1 -1], [1 -1]); 
>>den = conv(den,[1 -1]); 
>>initialvalue = polyval(num,1e20) / polyval (den, 1e20) 
initialvalue = 
2.0000e-20 


Therefore, fo = 0. 
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e Example 3.2.4 


Given the z-transform F(z) = (1 — a)z/|[(z — 1)(z — a)], where |z| > 1 > a > 0, then 
from the final-value theorem we have that 
l-a 


Jim fn lim (z — 1)F (z) = lim io 1. (3.2.42) 


This is consistent with the inverse transform fn = 1 — a” with n > 0. 
e Example 3.2.5 

Using the sequences f, = 1 and g, = a”, where a is real, verify the convolution 
theorem. 


We first compute the convolution of fn with gn, namely 


1 arti 


3.2.43 
l-a l-a ( ) 


n 
k 
Wn =fn* 9n => a = 
k=0 


Taking the z-transform of wn, 


z az 2 


osaa aaa. Gases oo. ees 


and the convolution theorem holds true for this special case. 


Problems 


Use the properties of z-transforms and Table 3.1.1 to find the z-transform of the following 
sequences. Then check your answer using MATLAB. 


0 n=0 

a —anT = , 

1. fa =nTe E ee n=l 
0, n=0 n 

are = oe n>1 4. fn = a” cos(n) 


[Hint : Use cos(n) = 4 (e + e~”)| 


5. fn = cos(n — 2)Hn-2 6. fn = 3 + e7?T 
0, n=0 
T. fn = sin(nwoT + 0) Se Oe, = 
. fn = sin(nwo j ie 2, ee fnt4 = fn 
1, n= 3, 
9 Fn = (=1)” 


(Hint: fn is a periodic sequence.) 


10. Using the property stated in Equation 3.2.20 and Equation 3.2.21 twice, find the z- 
transform of n? = n[n(1)”]. Then verify your result using MATLAB. 
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11. Verify the convolution theorem using the sequences fn = gn = 1. Then check your 
results using MATLAB. 


12. Verify the convolution theorem using the sequences fn = 1 and gn = n. Then check 
your results using MATLAB. 


13. Verify the convolution theorem using the sequences fn = gn = 1/(n!). Then check 
your results using MATLAB. Hint: Use the binomial theorem with x = 1 to evaluate the 
summation. 


14. If a is a real number, show that Z(a” fn) = F(z/a), where Z(f,) = F(z). 
3.3 INVERSE Z-TRANSFORMS 


In the previous two sections we dealt with finding the z-transform. In this section we 
find fn by inverting the z-transform F(z). There are four methods for finding the inverse: 
(1) power series, (2) recursion, (3) partial fractions, and (4) the residue method. We will 
discuss each technique individually. The first three apply only to those functions F(z) that 
are rational functions while the residue method is more general. For symbolic computations 
with MATLAB, you can use iztrans. 


| Power series 


By means of the long-division process, we can always rewrite F(z) as the Laurent 
expansion: 


F(z) = ao + a12 t tagz 7 +. (3.3.1) 


From the definition of the z-transform, 
F(z) =X faz” = fot fiz + faz? te, (3.3.2) 
n=0 


the desired sequence fn is given by an. 
e Example 3.3.1 


Let i N 
F(z) = = == foe (3.3.3) 


Using long division, N(z) is divided by D(z) and we obtain 


F(z)=htett+e%4+ e234 e tte. (3.3.4) 


Therefore, 


ao 5, a, = 1, a2 = 1, a3 = 1, ag = 1, etc., (3.3.5) 


which suggests that fo = 4 and fn = 1 for n > 1 is the inverse of F(z). 
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e Example 3.3.2 


Let us find the inverse of the z-transform: 


22? — 1.52 


Peja eee, 
(zl z2 — 1.5z + 0.5 


(3.3.6) 


By the long-division process, we have that 


2 + Seber + 1.25772? + 1.125773 + 
2—15z+0.5 | 22 — 1.5z 


227 3z + il 
1.5z — 1 
1.5z — 2.25 + 0.75027! 
1.25 — 0.750271 
1.25 — 1.87027! + 


1.12527! + 


Thus, fo = 2, fı = 1.5, fo = 1.25, f3 = 1.125, and so forth, or fn = 1+ (5)”. In general, 
this technique only produces numerical values for some of the elements of the sequence. 
Note also that our long division must always yield the power series Equation 3.3.1 in order 
for this method to be of any use. 

To check our answer using MATLAB, we type the commands: 
syms Z; syms n positive 
iztrans((2*z^2 - 1.5*z)/(z*2 - 1.5*z + 0.5),z,n) 
which yields 


ans = 
1 + (1/2)7n 


| Recursive method 


An alternative to long division was suggested® several years ago. It obtains the inverse 
recursively. 
We begin by assuming that the z-transform is of the form 


agz™ + apz™—! + age? +++! + am_12 + Om 
= ‘eat ? 
boz™ + byz™ 14 boz™ 2 E bm—12 f bm 


F(z) = (3.3.7) 


where some of the coefficients a; and b; may be zero and bo 4 0. Applying the initial-value 
theorem, 


fo = lim F(z) = ao/bo. (3.3.8) 


3 Jury, E. I., 1964: Theory and Application of the z-Transform Method. John Wiley & Sons, p. 41; 
Pierre, D. A., 1963: A tabular algorithm for z-transform inversion. Control Engng., 10(9), 110-111; 
Jenkins, L. B., 1967: A useful recursive form for obtaining inverse z-transforms. Proc. IEEE, 55, 574-575. 
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Next, we apply the initial-value theorem to z[F'(z) — fo] and find that 


fi = lim 2[F(2) — fol aon 
: (ao — bo fo)2z™ + (a1 — bı fo)! +--+ + (am — bm fo) 

a 3.3.10 

ain boz™ + bp z™—-1 + boz™—2 +e + bm-12 + bm 

ere (3.3.11) 


Note that the coefficient ao — bo fo = 0 from Equation 3.3.8. Similarly, 


fo= jim z|zF(z)— zfo— fı] (3.3.12) 
= (ao — bo fo)z™ t! + (a1 — bı fo — bo fi)z™ + (a2 — bo fo — bi f1)! +++ — bmfi 
= hm z 
z=>00 boz™ + by z™—! + baz? + --- + bm-12 + bm 
(3.3.13) 
= (a2 — be fo — bı fi)/bo (3.3.14) 


because ao — bo fo = ai — bı fo — fibo = 0. Continuing this process, we finally have that 
Fn = (an — bn fo — bni fi — +++ — bi fn—1) /bo, (3.3.15) 
where an = bn = 0 for n >m. 
e Example 3.3.3 
Let us redo Example 3.3.2 using the recursive method. Comparing Equation 3.3.7 to 


Equation 3.3.6, ao = 2, a, = —1.5, ag = 0, bọ = 1, bı = —1.5, bo = 0.5, and a, = bn = 0 if 
n > 3. From Equation 3.3.15, 


fo = ao/bo = 2/1 = 2, (3.3.16) 
fo = (a2 — be fo — bi fi) /bo (3.3.18) 
= [0 — (0.5)(2) — (-1.5)(1.5)]/1 = 1.25, (3.3.19) 
and 
fs = (a3 — b3 fo — b2 fı — bı f2) /bo (3.3.20) 
= [0 — (0)(2) — (0.5)(1.5) — (—1.5)(1.25)]/1 = 1.125. (3.3.21) 


| Partial fraction expansion 


One of the popular methods for inverting Laplace transforms is partial fractions. A 
similar, but slightly different, scheme works here. 
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e Example 3.3.4 


Given F(z) = z/(z? — 1), let us find fn. The first step is to obtain the partial fraction 
expansion of F(z)/z. Why we want F(z)/z rather than F(z) will be made clear in a 
moment. Thus, 


F(z) _ 1 _ A,B ead 
z  (z-1)\(z+1) 251° 241 
where 
F 1 
peepee a. abs Pea aA (3.3.23) 
a z=1 2 z=-1 
Multiplying Equation 3.3.22 by z, 
ees te 4 (3.3.24) 
a ON gen geet i 
Next, we find the inverse z-transform of z/(z — 1) and z/(z + 1) in Table 3.1.1. This 
yields 
-1 Z z -1 Z — (_1)n 
TOPE (ec ena 


Thus, the inverse is 


fn = 3 [L - (-1)"], n= 0. (3.3.26) 


From this example it is clear that there are two steps: (1) obtain the partial fraction 
expansion of F'(z)/z, and (2) find the inverse z-transform by referring to Table 3.1.1. 


e Example 3.3.5 
Given F(z) = 227/|(z + 2)(z+1)?], let us find fn. We begin by expanding F(z)/z as 


F(z) 2z A B C 


z ~ (+2(2+)l2 z+2 Pel ae (3.3.27) 
where 
ENS) 6 =-4, Bat E ye] =4, (3.3.28) 
a z=—2 dz Zz weed 
and A 
C=(241? 2 2, (3.3.29) 
z pty 
so that i , Í 
Z FA Z 
F(z) = z+1 z+2 (z+1) (3.3.30) 
or 


pes 2 zol zahl (3.3.31) 
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From Table 3.1.1, 


za(-2) Sei, z5) = (—2)”, (3.3.32) 
= zl] = zly l = —n(—-1)" = n(-1)™*1. (3.3.33) 


Applying Equation 3.3.32 and Equation 3.3.33 to Equation 3.3.31, 


fn = 4(-1)” — 4(—2)" + 2n(-1)", n > 0. (3.3.34) 


e Example 3.3.6 
Given F(z) = (2? + z)/(z — 2)?, let us determine fn. Because 


F(z) _z+1 1 3 
z “Sec 28 Ga (3.3.35) 


a a af $ z| (3.3.36) 


Referring to Table 3.1.1, 


z 3z 
go =) d Z|] = fnr. 3.3.37 
(A)r a l P 
Substituting Equation 3.3.37 into Equation 3.3.36 yields 


fa = (Èn +1) 2", n> 0. (3.3.38) 


| Residue method 


The power series, recursive, and partial fraction expansion methods are rather limited. 
We now prove that fa may be computed from the following inverse integral formula: 


1 
T) 2"T1F(z)dz, n20, (3.3.39) 
C 


h= 271 

where C is any simple curve, taken in the positive sense, that encloses all of the singularities 
of F(z). It is readily shown that the power series and partial fraction methods are special 
cases of the residue method. 
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Proof: Starting with the definition of the z-transform 
Baa faz”, |> R, (3.3.40) 
n=0 


we multiply Equation 3.3.40 by z”~! and integrating both sides around any contour C that 
includes all of the singularities: 


1 F = 1 dz 
— "—"F(z)dz= ma Pee a 3.3.41 
271 Jo á lakaz > f 201 f, 7 z (3.3.41) 


Let C be a circle of radius R, where R > R,. Then, changing variables to z = Ret’, and 
dz = iz d0, 


1 dz Reo ™ 20 , 1 pen 
Gay n-m”? _— i(n=m)0 = , , 
2ri So f z Or i $ ae { 0, otherwise. (3.3.42) 


Substituting Equation 3.3.42 into Equation 3.3.41 yields the desired result that 


1 
201 


f PFW) de = fn (3.3.43) 
C 


We can easily evaluate the inversion integral, Equation 3.3.39, using Cauchy’s residue 
theorem. 


e Example 3.3.7 


Let us find the inverse z-transform of 


F(z) = ; 3.3.44 
From the inversion integral, 
1 Bi 
n= dz. 3.3.45 
f nbr” ( ) 


Clearly the integral has simple poles at z = 1 and z = 2. However, when n = 0 we also have 
a simple pole at z = 0. Thus the cases n = 0 and n > 0 must be considered separately. 


Case 1: n = 0. The residue theorem yields 


1 1 1 
fo = Res| zea? Res il Res ey? | 
(3.3.46) 
Evaluating these residues, 


Res | 
Z 


1 
i] (z —1)(z —2) = (3.3.47) 
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Res| (3.3.48) 


1 
2(z— Teer 2(z — 2) 


1 1 i 
Bel yey =G- Dl 2 (3.3.49) 


Substituting Equation 3.3.47 through Equation 3.3.49 into Equation 3.3.46 yields fo = 0. 


Z=2 


Case 2: n > 0. Here we only have contributions from z = 1 and z = 2. 


gn-l gn-l 
n = Res| ——_______; 1] + Res] ——-_—___; 2 , > 0, 3.3.50 
n= Res Tea] +R E e i RERS 
where 
n—1 gn-l 
R = =-1 3.3.51 
“la Tc a G 
and 
| ies | die 1 0 (3.3.52) 
Res 2 i Naar |e 3. 
(z —1)(z- 2) ee lg es 
Thus, 
foot S4, n>0. (3.3.53) 
Combining our results, 
0, n=0, 
fn { LQ" 2),  n>0 (3.3.54) 
e Example 3.3.8 
Let us use the inversion integral to find the inverse of 
F(z) abel (3.3.55) 
z= : 3. 
(z— 1)? 
The inversion theorem gives 
1 gett ogn gett 4 Qgn 
r= dz = Res| ———_—: 1] , 3.3.56 
; mf gar | G-I ed 


where the pole at z = 1 is second order. Consequently, the corresponding residue is 


gett 4 2g” d 
| gape | A í ) 


=3n+1. (3.3.57) 


z=1 
Thus, the inverse z-transform of Equation 3.3.55 is 


fn =38n+1, n>0. (3.3.58) 
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e Example 3.3.9 


Let F(z) be a z-transform whose poles lie within the unit circle |z| = 1. Then 


Dose faz", Wied, (3.3.59) 
n=0 


and 


F(z -5 f+ 2 2 fie: (3.3.60) 


ném 


We now multiply both sides of Equation 3.3.60 by z~! and integrate around the unit circle 
C. Therefore, 


f F(z)F(27})z7} dz = Sf foz dz # S fnb f, z™="—l dz, (3.3.61) 
|z|=1 n=o0? |2l= 


n=0 m=0 |=1 
nm 


after interchanging the order of integration and summation. Performing the integration, 


2 lesa F(z)F(27*)z7" dz, (3.3.62) 


-5 |z|=1 


which is Parseval’s theorem for one-sided z-transforms. Recall that there are similar theo- 
rems for Fourier series and transforms. 


e Example 3.3.10: Evaluation of partial summations* 


We begin by noting that 


N N 


Se= fa= L F(z) 27t dz. (3.3.63) 


n=1 


Here we employed the inversion integral to replace fan and reversed the order of integration 
and summation. This interchange is permissible since we only have a partial summation. 
Because the summation in Equation 3.3.63 is a geometric series, we have the final result 


that 
o1 F(z)(z% = 1) 
SN = maf, SSi dz. (3.3.64) 


Therefore, we can use the residue theorem and z-transforms to evaluate partial summations. 


4 See Bunch, K. J., W. N. Cain, and R. W. Grow, 1990: The z-transform method of evaluating partial 
summations in closed form. J. Phys. A, 23, L1213-L1215. 
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Let us find Sy = S>*_,n3. Because fa = n3, F(z) = z(2? + 4z + 1)/(z — 1)*. 
Consequently 


2(z7 +424 1)(z% — 1) 1 d 
Sn = Res F = 15 : = ae [2(z? L Ago 1)(2% = 1)] (3.3.65) 
: z=1 
1 dt 
=F ae eg IEG. GNM gh ag ah m = 1(N +1)2N?. (3.3.66) 


e Example 3.3.11 


An additional benefit of understanding inversion by the residue method is the ability 
to qualitatively anticipate the inverse by knowing the location of the poles of F(z). This 
intuition is important because many engineering analyses discuss stability and performance 
entirely in terms of the properties of the system’s z-transform. In Figure 3.3.1 we graphed 
the location of the poles of F(z) and the corresponding fn. The student should go through 
the mental exercise of connecting the two pictures. 


Problems 


Use the power series or recursive method to compute the first few values of f» of the 
following z-transforms. Then check your answers with MATLAB. 


2 
; A ; 1 
L a ree + 0.9z + 0.09 FS z+ 
12.622 — 242 + 11.4 224 — 22° + 2z — 2 
1. 1. 
3. F(z) 5z/ + 1.5z & F(z) 62° + 6z 


~ 15.2522 — 36.757 + 30.75 ~ 1923 — 3322 + 21z—7 


Use partial fractions to find the inverse of the following z-transforms. Then verify your 
answers with MATLAB. 


E z(z +1) Az (1—e72?)z 
RS (@-DG?-2+1/4) ee) GG ae 
7. A EE 8. A oo) 


9. Using the property that the z-transform of gn = fn-kHn-k if n > 0 is G(z) = z7} F(z), 


find the inverse of 
z+1 


PO = mG 


Then check your answer with MATLAB. 


Use the residue method to find the inverse z-transform of the following z-transforms. Then 
verify your answer with MATLAB. 


z2 + 3z z 
C12} 
z 


CHIPE- I) 


10. F(z) = 


12. F(z) = 
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Figure 3.3.1: The correspondence between the location of the simple poles of the z-transform F(z) and 
the behavior of fn. 
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3.4 SOLUTION OF DIFFERENCE EQUATIONS 

Having reached the point where we can take a z-transform and then find its inverse, 
we are ready to use it to solve difference equations. The procedure parallels that of solving 
ordinary differential equations by Laplace transforms. Essentially we reduce the difference 
equation to an algebraic problem. We then find the solution by inverting Y (z). 
e Example 3.4.1 

Let us solve the second-order difference equation 


2Yn+2 — 3Yn41 + Yn = 5 3”, n > 0, (3.4.1) 


where yo = 0 and y; = 1. 
Taking the z-transform of both sides of Equation 3.4.1, we obtain 


22(yn+2) — 32(Ynsi) + Z(yn) = 5 Z(3”). (3.4.2) 


From the shifting theorem and Table 3.1.1, 


5 
22°Y (z) — 22°yo — 2zy1 — 3[z¥(z) — zyo] + Y (z) = ; i (3.4.3) 
Substituting yo = 0 and yı = 1 into Equation 3.4.3 and simplifying yields 
z(2z — 1) z 
2z — 1)(z — 1)Y (z) = =——— Y(z) = — =. 3.4.4 
B-E- DY) =E, œ Yos (3.4.4) 


To obtain yn from Y (z) we can employ partial fractions or the residue method. Applying 


partial fractions gives 
Y (z) A B 
= | (3.4.5) 


where 


1 
=5 (3.4.6) 


1 z 1 z 1 z 1 z 
Y(z)= n=—-~=Z' Zs, . (3.4. 
SE e E 2 (5) (5) pat) 
From Equation 3.4.7 and Table 3.1.1, 


Yn = 4 (3” — 1), n>0. (3.4.8) 


An alternative to this hand calculation is to use MATLAB’s ztrans and iztrans to 
solve difference equations. In the present case, the MATLAB script would read 
clear 
% define symbolic variables 
syms z Y; syms n positive 
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% take z-transform of left side of difference equation 
LHS = ztrans(2*sym(’y(n+2)’)-3*sym(’?y(n+1)’)+sym(’?y(n)’) ,n,z); 
% take z-transform of right side of difference equation 
RHS = 5 * ztrans(3°n,n,z); 
% set Y for z-transform of y and introduce initial conditions 
newLHS = subs(LHS, ’ztrans(y(n),n,z)’,’y(0)’,’y(1)’,Y,0,1); 
% solve for Y 
Y = solve (newLHS-RHS,Y) ; 
% invert z-transform and find y(n) 
y = iztrans(Y,z,n) 
This script produced 
y = 
-1/2+1/2*3^n 
Two checks confirm that we have the correct solution. First, our solution must satisfy 
the initial values of the sequence. Computing yo and yı, 


yo = 4(3° — 1) = 4(1 — 1) =0, and yı = 3(3'-1)=5(3-l1)=1. (3.4.9) 


Thus, our solution gives the correct initial values. 
Our sequence yn must also satisfy the difference equation. Now 


Yn = 4(3 +? —1) = 5(93"-1), and = Yn = § (38°F 1-1) = §.(3 3” —1). (3.4.10) 


Therefore, 
2Yyn+2 — 3Yn+1 + Yn = (9- $4+5)3"-14+3-5=53" (3.4.11) 


and our solution is correct. 

Finally, we note that the term 3” /2 is necessary to give the right side of Equation 3.4.1; 
it is the particular solution. The —1/2 term is necessary so that the sequence satisfies the 
initial values; it is the complementary solution. 


e Example 3.4.2 
Let us find the y, in the difference equation 
Yn+2 — 2Yn4i tyr =l, n20 (3.4.12) 


with the initial conditions yo = 0 and yı = 3/2. 
From Equation 3.4.12, 


Z(yn+2) — 2Z(yn+1) + Z (Yn) = Z(1). (3.4.13) 


The z-transform of the left side of Equation 3.4.13 is obtained from the shifting theorem, 
and Table 3.1.1 yields Z(1). Thus, 


z 


2°Y (z) — 2?yo — zy, — 2zY (z) + 2zyo + Y(z) = T (3.4.14) 
Pa 
Substituting yo = 0 and yı = 3/2 in Equation 3.4.14 and simplifying gives 
327 — z 327 — 2 
Y(z) = ——— el 3.4.1 
Osea ~ mme [eI ney 


The Z-Transform 185 


We find the inverse z-transform of Equation 3.4.15 by the residue method, or 


1 Zart _ zn 1 de [3z™*1 gn 
n= dz = = in? +n. 3.4.16 
z mf, 2—1 ol =| 2 =] ee ( ) 
Thus, 
Yn = 4n? +n, n>0. (3.4.17) 


Note that n?/2 gives the particular solution to Equation 3.4.12, while n is there so that 
Yn satisfies the initial conditions. This problem is particularly interesting because our 
constant forcing produces a response that grows as n?, just as in the case of resonance in 
a time-continuous system when a finite forcing such as sin(wọt) results in a response whose 
amplitude grows as t™. 


e Example 3.4.3 
Let us solve the difference equation 
b’Yn + Yn+2 = 9, (3.4.18) 


where the initial conditions are yo = b? and yı = 0. 
We begin by taking the z-transform of each term in Equation 3.4.18. This yields 


b’ Z (yn) + Z(Yn42) = 0. (3.4.19) 
From the shifting theorem, it follows that 
bY (z) +2°Y (2) — 2yo — zy = 0. (3.4.20) 
Substituting yo = b? and yı = 0 into Equation 3.4.20, 


b? 2? 


bY (2) +2°¥(z)—22=0, or Yh) = Sop. 


(3.4.21) 


To find yn we employ the residue method or 


1 b2 n+1 
Yn f Žž dz. (3.4.22) 
c( 


~ Oni z — ib)(z + ib) 
Thus, 
pezntl pezntt pr+2jn pnt2(_yn 
PO RAE an robl a 2 (3828) 
= a | — = prt? cos(“*) ; (3.4.24) 


because cos(x) = 4 (e’” + e~*”). Consequently, we obtain the desired result that 


yn = brt? cos(“*) for n > 0. (3.4.25) 


186 Advanced Engineering Mathematics: A Second Course 


oO 


9 
Sa aec0000000000000G0a00g0 
x 
— o 
E7 
5 o 
8 6 a 
© o Sua 
s5 g 2 o 

o o 
T4 Ki a o 
Boa 0° s 
3 a a 2 
o o 
E o o a 
a? T ° 
ae o a 
1 
a a 
0 1 1 1 1 1 
0 5 10 15 20 25 30 


number of conversion periods 


Figure 3.4.1: The amount in a savings account as a function of an annual conversion period when interest 
is compounded at the annual rate of 12% and $1000 is taken from the account every period starting with 
period 10. 


e Example 3.4.4: Compound interest 


Difference equations arise in finance because the increase or decrease in an account 
occurs in discrete steps. For example, the amount of money in a compound interest savings 
account after n + 1 conversion periods (the time period between interest payments) is 


Yn+1 = Yn +TYn, (3.4.26) 


where r is the interest rate per conversion period. The second term on the right side of 
Equation 3.4.32 is the amount of interest paid at the end of each period. 

Let us ask a somewhat more difficult question of how much money we will have if we 
withdraw the amount A at the end of every period starting after the period /. Now the 
difference equation reads 

Yn+1 = Yn +7TYn — AHn-e-1. (3.4.27) 


Taking the z-transform of Equation 3.4.27, 


Az E 
zY (z) — zyo = (1 + r)Y (z) — i (3.4.28) 
z= 
after using Equation 3.2.9 or 
Yoz Az 
Y(z)= ; 3.4.29 
= Ge eE] ( ) 
Taking the inverse of Equation 3.4.29, 
A 
m= yol)" A faery 1] Hae (3.4.30) 


The first term in Equation 3.4.30 represents the growth of money by compound interest 
while the second term gives the depletion of the account by withdrawals. 

Figure 3.4.1 gives the values of yn for various starting amounts assuming an annual 
conversion period with r = 0.12, £ = 10 years, and A = $1000. These computations were 
done in two ways using MATLAB as follows: 
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% load in parameters 
clear; r = 0.12; A= 1; k = 0:30; 
y = zeros(length(k) ,3); yanswer = zeros(length(k) ,3); 
% set initial condition 
for m=1:3 
y(1,m) = m; 
% compute other y values 
for n = 1:30 
y(nti,m) = y(n,m)+r*y(n,m); 
y(nti,m) = y(nt1,m)-A*stepfun(n, 11); 


end 
% now use Equation 3.4.30 
for n = 1:31 


yanswer(n,m) = y(1,m)*(1+r)*(n-1); 
yanswer(n,m) = yanswer(n,m)-A*((1+r) *(n-10)-1) 
*stepfun(n,11)/r; 
end; end; 
plot(k,y,’o’); hold; plot(k,yanswer,’s’); 
axis([0 30 0 10]) 
xlabel(’number of conversion periods’ ,’Fontsize’ ,20) 
ylabel(’amount left in account (K$)’,’Fontsize’ ,20) 


Figure 3.4.1 shows that if an investor places an initial amount of $3000 in an account 
bearing 12% annually, after 10 years he can withdraw $1000 annually, essentially forever. 
This is because the amount that he removes every year is replaced by the interest on the 
funds that remain in the account. 


e Example 3.4.5 
Let us solve the following system of difference equations: 
In41 = lln + Yn, and Yn+1 = 3%n + 3yn, (3.4.31) 


with the initial values of zo = 0 and yo = 5. 
Taking the z-transform of Equation 3.4.31, 


zX (z) — zoz =4X(z) + 2Y (z), zY (z) — yoz = 3X (z) + 3Y (z), (3.4.32) 


k (z—4)X(z)— 2Y (z) = 0, 3X (z) — (z — 3)Y (z) = —5z. (3.4.33) 


Solving for X (z) and Y (2z), 


10z 2z 2z 
me) (z — 6)(z — 1) T z=6 gee? Caen) 
a seg =) 2 3 
TOS paoe ge aT (3.4.35) 


Taking the inverse of Equation 3.4.34 and Equation 3.4.35 term by term, 


Ln = —2 +2 6”, and Yn = 342 6”. (3.4.36) 
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We can also check our work using the MATLAB script 


clear 

% define symbolic variables 

syms X Y z; syms n positive 

% take z-transform of left side of differential equations 

LHS1 = ztrans(sym(’x(n+1)’)-4*sym(’x(n)’)-2*sym(’y(n)’) ,n,z); 

LHS2 = ztrans(sym(’y(n+1)’)-3*sym(’x(n)’)-3*sym(’y(n)’) ,n,z); 

% set X and Y for the z-transform of x and y 

% and introduce initial conditions 

newLHS1 = subs(LHS1, ’ztrans(x(n) ,n,z)’,’ztrans(y(n),n,z)’,... 
>x(0)’,’?y(0)’,X,Y,0,5); 

newLHS2 = subs(LHS2, ’ztrans(x(n) ,n,z)’,’ztrans(y(n),n,z)’,... 
>x(0)’,’?y(0)’,X,Y,0,5); 

% solve for X and Y 

[X,Y] = solve(newLHS1,newLHS2,X,Y); 

% invert z-transform and find x(n) and y(n) 

x = iztrans(X,z,n) 

y = iztrans(Y,z,n) 

This script yields 

x = 

2*67n-2 

y = 

2*67nt+3 


Problems 


Solve the following difference equations using z-transforms, where n > 0. Check your answer 
using MATLAB. 


l. Yny Ye He Yo 2. Yn+2 — Yny tyn=0, y=y=l. 


3. Yn+2 — 2Yn41 +Yn =1, yum =Q. 4. Yntit3yn=n, yo=O0. 


5. Yn+1 — 5Yn = cos(nt), yo = 0. 6. Ynt2-44¥n =1, yul,y =0. 
T. Yn+2 — Gn = (5), Yo = Yı = 0. 8. Yn+2 — 5Ynti + 6Yn = 0, Yo = Yı = 1. 
9. Yn+2 — 3Yn+1ı + 2yn =1, yo=y1=0. 10. yn+2— 2Yyn+1 t Yn =2, yo=0, yı =2. 


l1. p41 = 3£n — 4Yn, Yn+1 = 2%n — 3Yn, To = 3, Yo = 2. 
12. £n+1 = 2£n — lOyn, Yn+ı = —En — Yn; To = 3, Yo = —2. 


13. @n41 = Zn — 2Yn, Yn+ı = — Yn, To =-—l1, yo = —7. 


14. n41 = 4% — 5Yn, Yn+1 = Tn — 2Yn, To =6, Yo = 2. 
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3.5 STABILITY OF DISCRETE-TIME SYSTEMS 


When we discussed the solution of ordinary differential equations by Laplace trans- 
forms, we introduced the concept of transfer function and impulse response. In the case of 
discrete-time systems, similar considerations come into play. 

Consider the recursive system 


Yn = aiYn-1Hn-1 + a2yn-2Hn-2 Hay NZ 0, (3.5.1) 
where H,_» is the unit step function. It equals 0 for n < k and 1 for n > k. Equation 
3.5.1 is called a recursive system because future values of the sequence depend upon all of 
the previous values. At present, a; and az are free parameters that we shall vary. 

Using Equation 3.2.7, 
2°Y (z) — a1zY (z) — a2Y (z2) = 2? X (z), (3.5.2) 


or 


= = ; 3.5.3 
X(z) 2z?—aız—a2 ( ) 


As in the case of Laplace transforms, the ratio Y(z)/X(z) is the transfer function. The 
inverse of the transfer function gives the impulse response for our discrete-time system. 
This particular transfer function has two poles, namely 


2 

ay ay 
SS ef : 3.5.4 
21,2 5 \/ ri + a2 ( ) 


At this point, we consider three cases. 


Case 1: a?/4+ a < 0. In this case z; and z2 are complex conjugates. Let us write them 
as 21,2 = re=ol Then 


2 2 
GS (z— RNG — re` iwoT) ~ 32 — Op Aaa +r?’ (Ae 
where r? = —ay, and woT = cos™! (a; /2r). From the inversion integral, 
gett gett 
Gn = Res È — 2r cos(woT)z + r? ' a| ERES È — 2r cos(woT)z + r? 72> (3:5:6) 
where gn denotes the impulse response. Now 
n+1 = n+l 
È — 2r ee Fr?’ a| = i aS re, 
Lp eplin + D BE adi + 1)woT] (3.5.8) 
etwol’ — e—iwo 2isin(woT) 
Similarly, i 
ne È — 2r ae Fr?’ > ae eoi an i (2:00) 
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and 
— r™sin[(n + 1)woT] 


n= 3.5.10 
sin(woT) ( ) 


A graph of sin[(n + 1)woT]/sin(woT) with respect to n gives a sinusoidal envelope. 
More importantly, if |r| < 1 these oscillations vanish as n —> oo and the system is stable. 
On the other hand, if |r| > 1 the oscillations grow without bound as n — oo and the system 
is unstable. 

Recall that |r| > 1 corresponds to poles that lie outside the unit circle while |r| < 1 is 
exactly the opposite. Our example suggests that for discrete-time systems to be stable, all 
of the poles of the transfer function must lie within the unit circle while an unstable system 
has at least one pole that lies outside of this circle. 


Case 2: a?/4+ a > 0. This case leads to two real roots, z1 and z2. From the inversion 
integral, the sum of the residues gives the impulse response 


In = 1. (3.5.11) 


Once again, if the poles lie within the unit circle, |z1| < 1 and |z2| < 1, the system is stable. 


Case 3: af /4+ az = 0. This case yields 21 = 22, 


2 1 gut aj” 
G(z) = Caa and ë Jn = ani f, a- mae = (2) (n+1). (3.5.12) 


This system is obviously stable if |a1/2| < 1 and the pole of the transfer function lies within 
the unit circle. 

In summary, finding the transfer function of a discrete-time system is important in 
determining its stability. Because the location of the poles of G(z) determines the response 
of the system, a stable system has all of its poles within the unit circle. Conversely, if 
any of the poles of G(z) lie outside of the unit circle, the system is unstable. Finally, if 
limno Jn = c, the system is marginally stable. For example, if G(z) has simple poles, 
some of the poles must lie on the unit circle. 


e Example 3.5.1 


Numerical methods of integration provide some of the simplest, yet most important, 
difference equations in the literature. In this example,” we show how z-transforms can be 
used to highlight the strengths and weaknesses of such schemes. 

Consider the trapezoidal integration rule in numerical analysis. The integral y, is 
updated by adding the latest trapezoidal approximation of the continuous curve. Thus, the 
integral is computed by 


where T is the interval between evaluations of the integrand. 


5 See Salzer, J. M., 1954: Frequency analysis of digital computers operating in real time. Proc. IRE, 
42, 457—466. 
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We first determine the stability of this rule because it is of little value if it is not stable. 
Using Equation 3.2.7, the transfer function is 


G(z) = os = 5 (: us r) ; (3.5.14) 


To find the impulse response, we use the inversion integral and find that 


T 1 
pay (3.5.15) 


At this point, we must consider two cases: n = 0 and n > 0. For n = 0, 


T z+1 T z+l1 T 
ZER 0| +=R | 3.5.16 
a esl ZH] 2 ef | 2 ( ) 
For n > 0, 
T n-1(z41 
Gees T ED EE (3.5.17) 
2 z—1 


Therefore, the impulse response for this numerical scheme is gg = 4 and gn =T for n > 0. 
Note that this is a marginally stable system (the solution neither grows nor decays with n) 
because the pole associated with the transfer function lies on the unit circle. 

Having discovered that the system is not unstable, let us continue and explore some 


of its properties. Recall now that z = eT = eT if s = iw. Then the transfer function 
becomes she 
Tite” iT wT 
= - = t ; 3.5.1 
G(w) JI e T 5 C0 ( 5 ) (3.5.18) 


On the other hand, the transfer function of an ideal integrator is 1/s or —i/w. Thus, the 
trapezoidal rule has ideal phase but its shortcoming lies in its amplitude characteristic; it 
lies below the ideal integrator for 0 < wT < m. We show this behavior, along with that for 
Simpson’s one-third rule and Simpson’s three-eighths rule, in Figure 3.5.1. 

Figure 3.5.1 confirms the superiority of Simpson’s one-third rule over his three-eighths 
rule. The figure also shows that certain schemes are better at suppressing noise at higher 
frequencies, an effect not generally emphasized in numerical calculus but often important in 
system design. For example, the trapezoidal rule is inferior to all others at low frequencies 
but only to Simpson’s one-third rule at higher frequencies. Furthermore, the trapezoidal 
rule might actually be preferred, not only because of its simplicity but also because it 
attenuates at higher frequencies, thereby counteracting the effect of noise. 


e Example 3.5.2 


Given the transfer function 


z2 


(g—D@— 1/2)’ 


is this discrete-time system stable or marginally stable? 

This transfer function has two simple poles. The pole at z = 1/2 gives rise to a term 
that varies as (3)” in the impulse response, while the z = 1 pole gives a constant. Because 
this constant neither grows nor decays with n, the system is marginally stable. 


G(z) = (3.5.19) 
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Figure 3.5.1: Comparison of various quadrature formulas by ratios of their amplitudes to that of an ideal 
integrator. (From Salzer, J. M., 1954: Frequency analysis of digital computers operating in real time. Proc. 
IRE, 42, p. 463.) 


e Example 3.5.3 


In most cases the transfer function consists of a ratio of two polynomials. In this 
case we can use the MATLAB function filter to compute the impulse response as follows: 
Consider the Kronecker delta sequence, x) = 1, and £n = 0 for n > 0. From the definition 
of the z-transform, X(z) = 1. Therefore, if our input into filter is the Kronecker delta 
sequence, the output yn will be the impulse response since Y(z) = G(z). If the impulse 
response grows without bound as n increases, the system is unstable. If it goes to zero as 
n increases, the system is stable. If it remains constant, it is marginally stable. 

To illustrate this concept, the following MATLAB script finds the impulse response 
corresponding to the transfer function, Equation 3.5.19: 


% enter the coefficients of the numerator 

% of the transfer function, Equation 3.5.19 
num = [1 0 O]; 

% enter the coefficients of the denominator 

% of the transfer function, Equation 3.5.19 
den = [1 -1.5 0.5]; 

% create the Kronecker delta sequence 

x = [1 zeros(1,20)]; 

% find the impulse response 

y = filter(num,den,x); 

% plot impulse response 

plot(y,’o’), axis([0 20 0.5 2.5]) 

xlabel (’nt+1’,’Fontsize’ ,20) 

ylabel(’impulse response’, ’Fontsize’ ,20) 


Figure 3.5.2 shows the computed impulse response. The asymptotic limit is two, so the 
system is marginally stable, as we found before. 

We note in closing that the same procedure can be used to find the inverse of any 
z-transform that consists of a ratio of two polynomials. Here we simply set G(z) equal to 
the given z-transform and perform the same analysis. 
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Figure 3.5.2: The impulse response for a discrete system with a transform function given by Equation 
3.5.19. 


Problems 


For the following time-discrete systems, find the transfer function and determine whether 
the systems are unstable, marginally stable, or stable. Check your answer by graphing the 
impulse response using MATLAB. 


1. Yn = Yn—-1Hn-1 FTn 2. Yn = 2Yn ify, 1— Yn 2Hn 2 +F Tn 
3. Yn = 3Yn-1Hn-1 + Tn 4. Yn = 4Yn—2Hn—2 + Ln 
Further Readings 


Jury, E. I., 1964: Theory and Application of the z-Transform Method. John Wiley & Sons, 
330 pp. The classic text on z-transforms. 


LePage, W. R., 1980: Complex Variables and the Laplace Transform for Engineers. Dover, 
483 pp. Chapter 16 is on z-transforms. 


Taylor & Francis 
Taylor & Francis Group 


http://taylorandfrancis.com 


— exact Hilbert transform 
0.8; © computed Hilbert transform J 


—e— 


| 
| 
ON 


+ 


Chapter 4 
The Hilbert Transform 


In addition to the Fourier, Laplace, and z-transforms, there are many other linear 
transforms that have their own special niche in engineering. Examples include Hankel, 
Walsh, Radon, and Hartley transforms. In this chapter we consider the Hilbert transform, 
which is a commonly used technique for relating the real and imaginary parts of a spectral 
response, particularly in communication theory. 

We begin our study of Hilbert transforms by first defining them and then exploring 
their properties. Next, we develop the concept of the analytic signal. Finally, we explore 
a property of Hilbert transforms that is frequently applied to data analysis: the Kramers- 
Kronig relationship. 


4.1 DEFINITION 


In Chapter 3 we motivated the development of z-transforms by exploring the concept 
of the ideal sampler. In the case of Hilbert transforms, we introduce another fundamental 
operation, namely quadrature phase shifting or the ideal Hilbert transformer. This proce- 
dure does nothing more than shift the phase of all input frequency components by —7/2. 
Hilbert transformers are frequently used in communication systems and signal processing; 
examples include the generation of single-sideband modulated signals and radar and speech 
signal processing. 

Because a —7/2 phase shift is equivalent to multiplying the Fourier transform of a 
signal by e~’*/? = —i, and because phase shifting must be an odd function of frequency,! 


1 For a real function the phase of its Fourier transform must be an odd function of w. 
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the transfer function of the phase shifter is G(w) = —i sgn(w), where sgn(-) is defined by 


1, t > 0, 
sen(t) = 0, t=0, 
—1, t<0. 


In other words, if X(w) denotes the input spectrum to the phase shifter, the output spec- 
trum must be —i sgn(w)X(w). If the process is repeated, the total phase shift is —7, a 
complete phase reversal of all frequency components. The output spectrum then equals 
[—i sgn(w)|?X(w) = —X(w). This agrees with the notion of phase reversal because the 
output function is —2(t). 

Consider now the impulse response of the quadrature phase shifter, g(t) = F~1[G(w)]. 
From the definition of Fourier transforms, 


dG . 
—=-i tg(t)e “" dt 4.1.1 
oni f tae ta, (4.1.1) 

and ae 

(7 
t) =-F |=]. 4.1.2 
=i) (4.1.2) 
Since G’(w) = —2id(w), the corresponding impulse response is 
1 


gt) = “F246 (4) =— (4.1.3) 


at 


Consequently, if x(t) is the input to a quadrature phase shifter, the superposition integral 
gives the output time function as 


A eee ee a AG (4.1.4) 


We shall define %(t) as the Hilbert transform of x(t), although some authors use the negative 
of Equation 4.1.4 corresponding to a +7/2 phase shift. The transform Z(t) is also called 
the harmonic conjugate of x(t). 

In similar fashion, Z(t) is the Hilbert transform of the Hilbert transform of z(t) and 
corresponds to the output of two cascaded phase shifters. However, this output is known 
to be —2(t), so F t) = —a(t), and we arrive at the inverse Hilbert transform relationship 
that 


ae ee Jp PMT es (4.1.5) 


Taken together, x(t) and Z(t) are called a Hilbert pair. Hilbert pairs enjoy the unique 
property that x(t) + iz(t) is an analytic function.? 


2 For the proof, see Titchmarsh, E. C., 1948: Introduction to the Theory of Fourier Integrals. Oxford 
University Press, p. 125. 
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Descended from a Prussian middle-class family, David Hilbert (1862-1943) would make significant 
contributions in the fields of algebraic form, algebraic number theory, foundations of geometry, 
analysis, mathematical physics, and the foundations of mathematics. Hilbert transforms arose 
during his study of integral equations (Hilbert, D., 1912: Grundztige einer allgemeinen Theorie der 
linearen Integralgleichungen. Teubner, p. 75). (Portrait courtesy of Photo AKG, London, with 
permission.) 


Because of the singularity at 7 = t, the integrals in Equation 4.1.4 and Equation 4.1.5 
must be taken in the Cauchy principal value sense by approaching the singularity point 
from both sides, namely 


D (Oe | [ se dr + J f(r) arl, (4.1.6) 


so that the infinities to the right and left of r = t cancel each other. See Section 1.10. 
We also note that the Hilbert transform is basically a convolution and does not produce a 
change of domain; if x is a function of time, then Ẹ is also a function of time. This is quite 
different from what we encountered with Laplace or Fourier transforms. 

From its origin in phase shifting, Hilbert transforms of sinusoidal functions are trivial. 
Some examples are 


as, 


cos(wt + p) = cos (wt + y — $) = sgn(w) sin(wt + p). (4.1.7) 


Similarly, 


pe 


sin(wt + y) = —sgn(w) cos(wt + p), (4.1.8) 
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and Ar! Pee 
ewttie = —i sen(wje tr, (4.1.9) 


Thus, Hilbert transformation does not change the amplitude of sine or cosine but does 
change their phase by 7/2. 


e Example 4.1.1 


Let us apply the integral definition of the Hilbert transform, Equation 4.1.4, to find 
the Hilbert transform of sin(wt), w 4 0. 
From the definition, 


H|sin(wt)] = - i j sine) dr. (4.1.10) 
If x =t—T, then 
{sin (wt)] = a | a 2 dx = — cos(wt) sen(w). (4.1.11) 


e Example 4.1.2 


Let us compute the Hilbert transform of x(t) = sin(t)/(¢? + 1) from the definition of 
the Hilbert transform, Equation 4.1.4. 
From the definition, 


sin(r e7 


= sae ‘a 
= =cpv f (7) Han dr = +3|pv [ (@— 7)? +1) dr| . (4.1.12) 


Because of the singularity on the real axis at T = t, we treat the integrals in Equation 4.1.12 
in the sense of Cauchy principal value. 

To evaluate Equation 4.1.12, we convert it into a closed contour integration by in- 
troducing a semicircle Cpr of infinite radius in the upper half-plane. This yields a closed 
contour C, which consists of the real line plus this semicircle. Therefore, Equation 4.1.12 
can be rewritten 


py f -A+ dr = PV ? ae) dz J -AFI dz. (4.1.13) 


The second integral on the right side of Equation 4.1.13 vanishes by Equation 1.9.7. 
The evaluation of the closed integral in Equation 4.1.13 follows from the residue theo- 
rem. We have that 
(z — t) e et 


Res| ait S (t — z)(z2 +1) oe rasa (4.1.14) 


and 


el (z — i) e et 


ps LG pean Bea 


We do not have a contribution from z = —i because it lies outside of the closed contour. 


(4.1.15) 


Res | 
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15. 


16. 


The Hilbert Transform of Some Common Functions 


function, x(t) 


l, a<t<b 
0, otherwise 


sin(wt + p) 


cos(wt + y) 


ewttpr 


œ| =| 


At + ua 
t2 +a?’ 


sin(at)Ji(at), O<a 
sin(at)J, (bt), O<b<a 
cos(at)Ji(at), O<a 


cos(at)J, (bt), O<b<a 


arn —a<t<a 


0, otherwise 


sin(avt) H(t), 0<a 


Hilbert transform, x(t) 


—sgn(w) cos(wt + vy) 
sgn(w) sin(wt + y) 
—i sgn(w)et +r" 
—n0(t) 


t 
a(t? + a?) 


pt — Aa 
t? + a? 


t(1 +t?) 
V2(1+ t4) 


1 — cos(at) 
t 


e7! — cos(t) 
1+? 


— cos(at) Jı (at) 
— cos(at) Jn (bt) 
sin(at)Jı (at) 


sin(at) J, (at) 


t+vt?—-—a*, -o<t<-a 


t, —a<t<a 


t—vVt?-a*, a<t<o 


zay ltl, -œ <t <0 


— cos(avt), 0<t< œ 


199 


200 Advanced Engineering Mathematics: A Second Course 


Therefore, 


= e’7 nie’ me (t+) 
P = t i 4.1.16 
v| aeni 241 +1 ( ) 


Only one half of the value of the residue at z = t was included; this reflects the semicircular 
indentation around the singularity there. Substituting Equation 4.1.16 into Equation 4.1.12, 
we obtain the final result that 


(4.1.17) 


e Example 4.1.3 


Let us employ the relationship that the Fourier transform of Z(t) equals —i sgn(w) times 


the Fourier transform of x(t) to find the Hilbert transform of x(t) = e”, 
Because F (et) = yre™™/4, 
ç ; —w?/4 
X (w) = —iyr sgn(w)e : (4.1.18) 


Therefore, 


: 0 . oœ 
Z(t) = sal. eft 07/4 dw — a eft /4 di (4.1.19) 


e2itn—n” dn — i T 2itn=n° q 
n = e n (4.1.20) 
“vel T Jo 

ioo 2e —t? t 7 
e7 ds — = e? ds = ~— | e ds, 
= P We Vi 


where s = t+ ni. The integral in Equation 4.1.21 is the well-known Dawson’s integral. See 
Gautschi and Waldvogel* for an alternative derivation. 


(4.1.21) 


e Example 4.1.4: Numerical computation of the Hilbert transform 


Recently André Weideman? devised a particularly efficient method for numerically 
computing the Hilbert transform when z(t) is known exactly for any real t and enjoys the 
property that 


i. |x(t)|? dt < œ. (4.1.22) 


—oco 


3 Press, W. H., S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery, 1992: Numerical Recipes in 
Fortran: The Art of Scientific Computing. Cambridge University Press, Section 6.10. 


4 Gautschi, W., and J. Waldvogel, 2000: Computing the Hilbert transform of the generalized Laguerre 
and Hermite weight functions. BIT, 41, 490-503. 


5 Weideman, J. A. C., 1995: Computing the Hilbert transform on the real line. Math. Comput., 64, 
745-762. 
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Given Equation 4.1.22, the function x(t) can be represented by the rational expansion 


x(t) = 5 anpn(t), (4.1.23) 
where pn(t) is the set of rational functions 
(1+ it)” 
n(t) = an. =0,+1,+2,::-, 4.1.24 
pol) =a, =O (4.1.24) 
and E 
an = 1 x(t)ož (t) dt (4.1.25) 
T J- 
or T 
an = 5 J [1 — étan(1)] x[tan(26)] e~? ao, (4.1.26) 


if we introduce the substitution t = tan(0/2). 
Why is Equation 4.1.23 useful? Taking the Hilbert transform of both sides of this 


equation, 
Co 


P= So analt). (4.1.27) 
Using contour integration, we find that 
(1 +ir)” : 
lp dr = n(t), 4.1.28 
-irv faa i = i sell (4.1.28) 


where sgn(t) is the signum function with sgn(0) = 1. Therefore, 


Co 


P) =i X. sgn(n) an pr(t). (4.1.29) 


n=—Cco 


We must now approximate Equation 4.1.29 so that we can evaluate it numerically. We 
do this by introducing the following truncated version: 


N-1 


È —i X. sgn(n) An pn(t). (4.1.30) 


n=—N 


8 
Z 
I 


This particular truncation was chosen because pn (t) and p_y—1(t) are a conjugate pair. The 
coefficient a, has become An, which equals 


N-1 
1 X 1 p T —inĝ; 
An = N >, [l-itan(30;)] x[tan(50,)] 07", (4.1.31) 
j=-N+1 
where 6; = 7j/N. The terms corresponding to j = +N have been set to zero because it 


is assumed that x(t) vanishes rapidly with t > +00. Finally, we substitute 0 for t and 
transform Equation 4.1.30 into 


? N-1 
En(t;) = — init) ee sgn(n) Ane”, (4.1.32) 
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— exact Hilbert transform 
0.8; © computed Hilbert transform 4 


Figure 4.1.1: The Hilbert transform for z(t) = 1/(1 + t4) computed from Weideman’s algorithm. 


The advantage of Equation 4.1.31 and Equation 4.1.32 is that they can be evaluated 
using fast Fourier transforms. For example, the following MATLAB script devised by Wei- 
deman illustrates his methods for x(t) = 1/(1 + t4): 


% initialize parameters used in computation 

b = 1; N= 8; n = [-N:N-1]’; 

% set up collocation points and evaluate function there 
t = b * tan(pi*(n+1/2)/(2*N)); F = 1./(1+t.74); 

% evaluate Equation 4.1.31 

an = fftshift (fft(F.*(b-i*t))); 

% compute Hilbert transform via Equation 4.1.32 

hilbert = ifft(fftshift (i*(sign(nt+1/2) .*an)))./(b-i*t) ; 
hilbert = -real(hilbert) ; 

% find points at which we will compute exact answer 

tt = [-12:0.02:12]; 

% compute exact answer 

answer = tt.*(1+tt.°2)./(1+tt.*4) ./sqrt (2); 

fzero = zeros(size(tt)); 

% plot both computed Hilbert transform and exact answer 
plot(tt,answer,’-’,t,hilbert,’o’,tt,fzero,’--’) 
xlabel(’t’,’Fontsize’ ,20) 

legend(’exact Hilbert transform’ ,’ computed Hilbert transform’) 
legend boxoff 


Figure 4.1.1 illustrates Weideman’s algorithm for numerically computing the Hilbert 
transform of 1/(1 + t*). 

There are two important points concerning Weideman’s implementation of his algo- 
rithm. First, the collocation points originally given by t; = tan[7j/(2N)], j =—N,...,N-1 
have changed to t; = tan[(j + $)7/(2N)], j = —N,...,N—1. This change replaces the 
trapezoidal rule discretization for the Fourier coefficients with a midpoint rule. The advan- 
tages are twofold: First, it avoids the nuisance of dealing with a collocation point at infinity. 
Second, it actually yields more accurate results in many cases. 
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The discerning student will also notice that Weideman introduced a free parameter b, 
which we set to one. This rescaling parameter can have a major influence on the accuracy. 
The interested student is referred to the bottom of page 756 in Weideman’s paper for further 
details. 


e Example 4.1.5: Discrete Hilbert transform 


Quite often the function is given as discrete data points. How do we find the Hilbert 
transform in this case? We will now prove® that the equivalent discrete Hilbert transform 
is 


k even, 

HT fe 4 ge (4.1.33) 
-So—., k odd, 
T k-n 


where f» denotes a set of discrete data values that are sampled at t = nT and both k and 
n run from —co to oo. The corresponding inverse is 


r me 
= > a n even, 
fn = ae F (4.1.34) 
k 
= X dd. 
aN wee 


We begin our proof by inserting Equation 4.1.33 into Equation 4.1.34. For n even, 


2 1 2 z 4 , 
TiS li 2 E )-4 D D (4.1.35) 


Z peven kodd 


4 Jn, 4 1 1 
= eae S Dh Dhi —}: (4.1.36) 


peven,p#£n k odd 


The term within the curly brackets equals zero as k runs through all of its values. Therefore, 
Equation 4.1.36 reduces to 


8 1 1 1 
= Sh (it gt gttat). (4.1.37) 


However, the term in the brackets of Equation 4.1.37 equals 77/8. Therefore, Equation 
4.1.33 and Equation 4.1.34 is proved for n even. An identical proof follows for n odd. 

A popular alternative’ to Equation 4.1.33 involves the (fast) Fourier transform and 
the relationship that X(w) = —i sgn(w)X(w), where X(w) and X(w) denote the Fourier 
transform of x(t) and Z(t), respectively. In this technique, a fast Fourier transform is taken 
of the data. This transformed dataset is then multiplied by —i sgn(w) and then back 
transformed to give the Hilbert transform. 


6 See Kak, S. C., 1970: The discrete Hilbert transform. Proc. IEEE, 58, 585-586. For an alternative 
derivation, see Kress, R., and E. Martensen, 1970: Anwendung der Rechteckregel auf die reelle Hilbert- 
transformation mit unendlichem Intervall. Z. Angew. Math. Mech., 50, T61—T64. 


T Čížek, V., 1970: Discrete Hilbert transform. IEEE Trans. Audio Electroacoust., AU-18, 340-343. 
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Let x(t) be a real, even function. Then X(w), the Fourier transform of x(t), is also an 
even function. Consequently, 


x(t) = = ~ X (we? dw = > T —isgn(w)X (w) [cos(wt) + isin(wt)] dw (4.1.38) 

E =i y sgn(w)X (w) cos(wt) dw + ~ E sgn(w)X (w) sin(wt) dw (4.1.39) 
1° ; 

= | X (w) sin(wt) dw. (4.1.40) 


Note that the Hilbert transform in this case is an odd function. Similarly, if x(t) is a real, 
odd function, 


Oe -* I E (4.1.41) 
0 
and the Hilbert transform is an even function. 
Problems 
1. Show that the Hilbert transform of a constant function is zero. 
2. Use Equation 4.1.4 to compute the Hilbert transform of cos(wt), w 4 0. 


3. Use Equation 4.1.4 to show that the Hilbert transform of the Dirac delta function ô(t) 
is 1/(at). 


4. Use Equation 4.1.4 to show that the Hilbert transform of 1/(t? + 1) is t/(t? +1). 
5. The output y(t) from an ideal lowpass filter can be expressed by the convolution integral 


in(2rwt 
„ SP1 Tut) 


y(t) = x(t) 


at : 


where x(t) is the input signal. Show that this expression can also be expressed in terms of 
Hilbert transforms as 


y(t) = Hle (t) cos(2rwt)] sin(Qrwt) — H[x(t) sin(2rwt)] cos(2rwt). 


Following Example 4.1.3, find the Hilbert transforms of 


1, —a<t<a 
otherwise 


8. Using the commutative and associate properties of convolution, f(t)» g(t) = g(t) * f(t) 
and [f(t) * g(t)] * v(t) = f(t) x [g(t) * v(t)], respectively, and the definition of the Hilbert 
transform, Equation 4.1.4, show® that 


a 


Hf (t) * g(t)] = FE) * g) = FH * g(t). 


8 For an application, see Sakai, H., and G. A. Vanasse, 1966: Hilbert transform in Fourier spectroscopy. 
J. Opt. Soc. Am., 56, 131-132. 
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Using MATLAB, test Weideman’s algorithm for the following cases. Why does the algorithm 
do well or not? 


1, -1l<t<1 1 sin(t) 


9. l oeri 10. sin(t) 11. Pal 12. Ip 


For Problem 12, you will need 


tt +1 tt +1 


Pf | mo] -e~ Y?[cos(1/V2) + sin(1/v2 )t?] — cos(t) 


4.2 SOME USEFUL PROPERTIES 


In principle, we could construct any desired transform from the definition of the Hilbert 
transform. However, there are several general theorems that are much more effective in 
finding new transforms. 


| Linearity 


From the definition of the Hilbert transform, it immediately follows that if z(t) = 
c, x(t) + coy(t), where cı and c2 are arbitrary constants, then 2(t) = c(t) + coy(t). 


The energy in a signal and its Hilbert 
transform are the same. 


Consider the energy spectral densities at input and output of a quadrature phase shifter. 
The output equals 


[X(w)? = FEO? = |- i sen(w)?|X(w)/? = Xw). (4.2.1) 


Because the energy spectral density at input and output are the same, so are the total 
energies. 


A signal and its Hilbert transform 
are orthogonal. 


From Parseval’s theorem 
? 


N a(t)@(t) dt = ie X (w)X*(w) dw, (4.2.2) 


where X(w) = F[2(t)]. Then, 


ie X(w).X*(w) dw = T i sen(w)|X(w)|? dw = 0, (4.2.3) 
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because the integrand in the middle expression of Equation 4.2.3 is odd. Thus, 


/ T SOE dt = 0. (4.2.4) 


—oo 


The reason why a function and its Hilbert transform are orthogonal to each other follows 
from the fact that a Hilbert transformation of a function shifts the phase of each Fourier 
component of the function forward by 7/2 for positive frequencies and backward for negative 
frequencies. 


e Example 4.2.1 


Let us verify the orthogonality condition for Hilbert transforms using z(t) = 1/(1+t?). 
Because Z(t) = t/(1 + t?), 


ie 2(t)a(t) dt = i ELAO pE (4.2.5) 


—oo 


since the integrand is an odd function. 


| Shifting 


Let us find the Hilbert transform of x(t + a) if we know Z(t). From the definition of 
Hilbert transforms, 


gee SAS F zuta is J 7 sae — dr = H(t +0) (4.2.6) 


—Co 


or H|x(t + a)| = x(t + a). 


| Time scaling 


Let a > 0. Then, 


Lian = i. AON) gins T An: (4.2.7) 


œ t-n T 


On the other hand, if a < 0, 


H{x(at)| = ~ PAOD): ip Jp Aa a aay, (4.2.8) 


TG ove GA] Tbe ee Mb ST 


Thus, we have that H[x(at)] = sgn(a) Z(at). 
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Some General Properties of Hilbert Transforms 


function, x(t) Hilbert transform, x(t) 

1 x(t) —x(t) 
2 a(t) + y(t) a(t) +P) 
3 x(t+a), areal x(t + a) 

d” x(t) d” z(t) 
j dt” dt” 
5 x(at) sen(a) £(at) 
6 ta(t) t@(t) + 4 f° a(t) dr 
7 (t + a)x(t) (t+ a)@(t) + 2 fS a(t) dr 


| Derivatives 


Let us find the relationship between the nth derivative of x(t) and its Hilbert transform. 
Using the derivative rule as it applies to Fourier transforms, 


{| Fe] } = —isemw) ie)” = iisto] = (ea) Kw) = F | FE] 


dt” din 
(4.2.9) 
Taking the inverse Fourier transforms, we have that 
ae yan (4.2.10) 
dt” j gre -_ 


| Convolution 


Hilbert transforms enjoy a similar, but not identical, property with Fourier transforms 
with respect to convolution. If 


w(t) = u(t) x v(t) = J 


=p 


Co Co 


u(r)v(t — 7) dr = / u(t — T)v(r) dr, (4.2.11) 
then 
w(t) = v(t) x u(t). (4.2.12) 


Proof: From the convolution theorem for Fourier transforms, W(w) = V(w)U(w). Multi- 
plying both sides of the equation by —isgn(w), 


Ww) = —~isgn(w)W(w) = V(w)[-isgn(w)U(w)] = V (w)? (w). (4.2.13) 
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Again, using the convolution theorem as it applies to Fourier transforms, we arrive at the 
final result. 


e Example 4.2.2 


Given the functions u(t) = cos(t) and v(t) = 1/(1 + tt), let us verify the convolution 
theorem as it applies to Hilbert transforms. 


With u(t) = cos(t) and v(t) = 1/(14 tt), 


w(t) = u(t) * v(t) =f ost) ae (4.2.14) 
a = t cox 2) oe | ie Aaa dz (4.2.15) 
= ee VV? [co oe ee cos 
zT | +( 5) +s (=)| os(t) (4.2.16) 
so that 
a T -1/2 A 1 i ES in 
H(t) = z / feos( -) +s (=)| sin(t). (4.2.17) 
Because D(t) = t(1 + t?)/[/2 (1 + tt), 
u(t) x U( d=) cos(t — x) ute) ) dx (4.2.18) 
cos(t) cos(x)x(1 + x?) 1 /® sin(t) sin(a)a(1 + 2?) 
-zj 1+24 det f 1+ 24 a 
(4.2.19) 
eee eee © (1+ 27) sin(z) 
= aa D a4 _9 1 (4.2.20) 


and the convolution theorem for Hilbert transforms holds true in this case. 


| Product theorem 


Let f(t) and g(t) denote complex functions with Fourier transforms F(w) and G(w), 
respectively. If 


1) F(w) vanishes for |w| > a, and G(w) vanishes for |w| < a, where a > 0, 
or 
2) f(t) and g(t) are analytic functions (their real and imaginary parts are Hilbert pairs), 


then the Hilbert transform of the product of f(t) and g(t) is 


HIFO = fg). (4.2.22) 
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N 
u+v=0 u+v=0 
(a) F(u)=0,lul>a (b) F(u) = 0, u<0 
G(u) = 0, Ivl < a G(u) = 0, v<0 
Figure 4.2.1: Region of integration in the proof of the product theorem. 
Proof:? The product f(t)g(t) can be expressed as 
1 ii i i(u+v)t 
f()g(t) = Tg F(u)G(v)e dv du. (4.2.23) 
T —oo J — 00 
Because H(et®*) = i sgn(b)e®, 
HOD = zs / / F(u)G(v) sen(u+ vel“ du du. (4.2.24) 


The shaded regions of Figure 4.2.1 are those in which the product F'\(u)G(v) is nonvanishing 
for the conditions of the theorem. In Figure 4.2.1(a) the nonoverlapping Fourier transforms 
yield two semi-infinite strips in which the product is nonvanishing. In Figure 4.2.1(b), 
for analytic functions, the Fourier transforms vanish for negative arguments!? so that the 
product is nonvanishing only in the first quadrant. In both cases sgn(u + v) = sgn(v) over 
the regions of integration in which the integrand is nonvanishing. Thus, 


HOJA] = na / ‘i L * F(u)G(v) sen(v)el“ dy du (4.2.25) 
= opa n G(v) sgn(v)e™* du = f(t)g(t). (4.2.26) 


e Example 4.2.3: Hilbert transforms of band-pass functions 


In communications, we have the double-sideband, amplitude-modulated signal given 
by a(t) cos(wt + p), where y is constant. From the product theorem, its Hilbert transform 
equals a(t) sin(wt + p), w > 0, provided that the highest frequency component in a(t) is 
less than w. Paradoxically, the Hilbert transform of more general a(t) cos[wt + y(t)], which 
equals a(t) sin[wt + y(t)], has no such restriction. 


9 See Bedrosian, E., 1963: A product theorem for Hilbert transforms. Proc. IEEE, 51, 868-869. This 
theorem has been extended to functions of n-dimensional real vectors by Stark, H., 1971: An extension of 
the Hilbert transform product theorem. Proc. IEEE, 59, 1359-1360. 


10 Titchmarsh, E. C., 1948: Introduction to the Theory of Fourier Integrals. Oxford University Press, 
p. 128. 


210 Advanced Engineering Mathematics: A Second Course 


Problems 
Verify the orthogonality property of Hilbert transforms using 
1. x(t) =1/(1 +t) 2. x(t) = sin(t)/(1 + t?) 


; O0<t<a 
eee) t otherwise 


Verify the convolution theorem for Hilbert transforms using 


1, 0<t<a, ; 1 
4. u(t) = o Silena v(t) = sin(t) 5. u(t) = cos(t), v(t) = 


6. Use the product theorem to show that 
H|sin(at) J, (bt)] = — cos(at) Jn (bt), 0<b<a, 


if n = 0,1,2,3,.... 


Hint: ci el 
2(-1)™ Ww 
F|Jn(bt)] = Th H(b 
a(t] = Fa Ta( E 
where T,,(-) is a Chebyshev polynomial of the first kind and m = n/2 or (n—1)/2, depending 
upon which definition gives an integer. 


7. Given cosine and sine integrals: 


and 
H(Ci(alt|)] = —sgn(t)Si(alt|), O<a, 


use the product rule to show that 
H|sin(bt)Ci(a|t|)] = —sgn(t) sin(bt)Si(alt|),  O<b<a. 
Hint: 


0, 0 < |w| <a, 


0 <a. 
-rflwl, a< |u| <, 


F[Ci(alt))| = { 


8. Prove that 


Hint: 
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4.3 ANALYTIC SIGNALS 


The monochromatic signal A cos(wot + p) appears in many physical and engineering 
applications. It is common to represent this signal by the complex representation Ae’(ot+?) , 
These two representations are related to each other by 


Acos(wot + p) = R[ Aci ott#)] = 2 [Aet + emitt] (4.3.1) 


1 
Furthermore, the Fourier transform of A cos(wot + y) is 
F [A cos(wot + y)] = 5 [Aet S (w — wo) + Ae"? 5(w + wo)] , (4.3.2) 
while the Fourier transform of Ae“0'+?) is 
Fapte] = Aet? S (w — wo). (4.3.3) 


As Equation 4.3.2 and Equation 4.3.3 clearly show, in passing from the real signal to its 
complex representation, we double the strength of the positive frequencies and remove 
entirely the negative frequencies. 

Let us generalize these concepts to nonmonochromatic signals. For the real signal x(t) 
with Fourier transform X (w) and the complex signal z(t) with Fourier transform Z(w), the 
previous paragraph shows that our generalization must have the property: 


Z(w) = X(w) + sgn(w)X (w) (4.3.4) 
or 
2X (w), w > 0, 
Zw) = 4 X(w), w =0, (4.3.5) 
0, w <0. 


Taking the inverse of Equation 4.3.4, we have the definition of an analytic signal as 

z(t) = x(t) + i2?(t), (4.3.6) 
where x(t) is a real signal and Z(t) is its Hilbert transform. 
e Example 4.3.1 


In Figure 4.3.1 the amplitude spectrum of the analytic signal is graphed when x(t) is 
the rectangular pulse, 
2 fs, It] <a, 
oO T ltl > a. 


Note that the amplitude spectrum equals zero for w < 0 and twice the amplitude spectrum 
for w > 0. 


e Example 4.3.2 


Let us find the energy of an analytic signal. 
The energy of an analytic signal is 


T oas f OLES Z(t) dt = a at= f |X (w)|? dw (4.3.7) 


—oco —oCo =00 —0O —oo 
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5.0 y T T T 


amplitude /a 


phase (radians) 


-1.0 . : 
-20.0 -10.0 0.0 10.0 20.0 


wa 


Figure 4.3.1: The spectrum of the analytic signal when x(t) is the rectangular pulse given in Example 
4.3.1. 


Kol (a) A (b) 


Oax Oiz Wo 
IZ(a)| (c) A (a) 
| \. 
> > 
ax — % 


Figure 4.3.2: Given a function z(t) with an amplitude spectrum shown in (a), frame (b) shows the 
amplitude spectrum of the amplitude-modulated signal x(t) cos(wot) while frames (c) and (d) give the 
amplitude spectrum of the analytic signal z(t) and x(t) cos(wot) — T(t) sin(wot), respectively. 


by Parseval’s theorem. Thus, the analytic signal has twice the energy of the corresponding 
real signal. 


Consider the function x(t) whose amplitude spectrum is shown in Figure 4.3.2(a). If 
we were to amplitude modulate x(t) with cos(wot), then the amplitude spectrum of this 
modulated signal would appear as pictured in Figure 4.3.2(b). 

Consider now the signal 


y(t) a x(t) cos(wot) = T(t) sin(wot) = R{ [x(t) + i(t) t} (4.3.8) 
= R{2(t)e t} = 5 [eet + 2*(He"] , 


where z(t) is the analytic signal of x(t). We have plotted the amplitude spectrum |Z(w)| in 
Figure 4.3.2(c). If we computed the amplitude spectrum of y(t), we would find that 
Y(w) = 


tees Na upc (4.3.10) 


The Hilbert Transform 213 


X(w — wo), Wo < w < wo + Wmax, 
Y (w) = 4 X* (—w = wo), —W0 — Wmax SW < wo, (4.3.11) 
0, otherwise. 


We have sketched this amplitude spectrum |Y (w)| in Figure 4.3.2(d). Each triangular part 
is called the single sideband signal because it contains the upper frequencies (|w| > wo) of 
the modulated signal z(t) cos(wot). Similarly, if we had used z(t) cos(wot) + Z(t) sin(wot), 
we would only have obtained the lower sidebands. Consequently, a communication system 
using x(t) cos(wot) — Z(t) sin(wot) or x(t) cos(wot) + Z(t) sin(wot) would realize a 50% savings 
in its frequency bandwidth over one transmitting x(t) cos(woft). 


Problems 
1. Find the analytic signal corresponding to x(t) = cos(wt), w > 0. 


2. Show that the polar form of an analytic signal can be written 
a(t) = |2(z)le"?, 


where ay 
|z(t)|? = r? (t) +2? (t), y(t) = tan? Ea ; 


3. Analytic signals are often used with narrow-band waveforms with carrier frequency wo. 
If y(t) = wot + y'(t), show that the analytic signal can be written z(t) = r(t)e’“°t, where 
r(t) = |z(t)|e’ ®©. The function r(t) is called the complex envelope or the phasor amplitude; 
this is a generalization of the phasor idea beyond pure alternating currents. 


4.4 CAUSALITY: THE KRAMERS-KRONIG RELATIONSHIP 


Causality is the physical principle which states that an event cannot proceed its cause. 
In this section we explore what effect this principle has on Hilbert transforms. 

We begin by introducing the concept of causal functions. A causal function is a function 
that equals zero for all t < 0. As with all functions we can write it in terms of an even x(t) 
and an odd 2,(t) part as z(t) = x(t) + x(t). Because x(t) is causal, x(t) = sgn(t)a-(t) 
and 

x(t) = x(t) + sgn(t)x-(t). (4.4.1) 


Taking the Fourier transform of Equation 4.4.1, we find that the Fourier transform of all 
causal functions are of the form 


X(w) = Xe(w) — iĝelw), (4.4.2) 
where 
X.w) = J A. end Rie Ke(7) dr, (4.4.3) 
TJ wT TJ wT 
— wai aes [| ee (4.4.4) 
Aa TEES oT iv 


—oco 
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Equation 4.4.3 first arose in dielectric theory and, taken together, are called the Kramers!’ 


and Kronig'? relation after their discoverers, who derived these relationships during their 
work on the dispersion of light by gaseous atoms or molecules. 
e Example 4.4.1 


Let us verify the Kramers-Kronig relation using the causal time function x(t) = H(t). 


Because a(t) = 5 and X-(w) = 7d(w), 


Cease T PO aa, (4.4.5) 


T W— T W 


— 0o 
Consequently, by the Kramers-Kronig relation, 
i 


FIHO] = Xelw) — i£e(w) = rô(w) + —. (4.4.6) 


W 


e Example 4.4.2 


A simple example of a causal function is the impulse response or Green’s function intro- 
duced in earlier chapters. From Equation 4.4.2 we have the result that the transfer function 
G(w), the Fourier transform of the impulse response, must yield the Hilbert transform pair 
Ge (w) — iG. (w). 

For example, if g(t) = e~' H(t), then G(w) = 1/(1+ iw). Because 


eee an) (4.4.7) 
i Bel WHI T 
we have the Hilbert transform pair of 
1 Be t 
e Example 4.4.3 
Let us verify the Kramers-Kronig relation for the Hilbert transform pair 
1 a t(t? + 1) 
z(t) = ——— and z(t) = ——__— 4.4.9 
O=a5 Oman (4.4.9) 


by direct integration. 


11 Kramers, H. A., 1929: Die Dispersion und Absorption von Réntgenstrahlen. Phys. Z., 30, 522-523. 


12 Kronig, R. de L., 1926: On the theory of dispersion of x-rays. J. Opt. Soc. Am., 12, 547-551. 
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From Equation 4.4.3, we have that 


w(w? + 1) 1 L. dr 
——— = ——. 4.4.10 
Za Tj D-7) (E0) 
Applying the residue theorem to the right side of Equation 4.4.10, we obtain 
w(w? + 1) ; | 1 | ; | 1 i/4 
————. = iRes ;w| + 2i Res] ———__—__; e 
J(u + 1) CESOS G+- z) 
1 , 
2i Res | ——— ; e84] 4.4.11 
+2i slz" ( ) 


We only include one half of the value of the residue at 7 = w because the singularity lies 
on the path of integration and we must treat this integration along the lines of a Cauchy 
principal value. Evaluating the residues, we find 


Res oT = = (4.4.12) 
A E E, e] 7 al ara ; were 

and . 
Res| | = ae ge y . (4.4.14) 


Substituting Equation 4.4.12 through Equation 4.4.14 into the right side of Equation 4.4.11, 
we obtain the left side. 


Problems 
1. For a causal function z(t), prove that xo(t) = sgn(t)xe(t) and ze(t) = sgn(t)xo(t). 


2. Redo our analysis if x(t) is a negative time function, i.e., x(t) = 0 if t > 0. Verify your 
result using x(t) = e'H(—t). 


3. Using g(t) = te~* H(t), find the corresponding Hilbert transform pairs. 
4. Using g(t) = e™* cos(wt) H(t), find the corresponding Hilbert transform pairs. 


5. Verify the Kramers-Kronig relation for the Hilbert transform pair: 


1 t 
=> d z(t) = —— 
t +1 or e) t? +1 


z(t) 
by direct integration. 
Further Reading 


Hahn, S. L., 1996: Hilbert Transforms in Signal Processing. Artech House, 442 pp. Covers 
the basic theory and gives some practical applications. 


Taylor & Francis 
Taylor & Francis Group 


http://taylorandfrancis.com 
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Chapter 5 
Green’s Functions 


An important aspect of engineering mathematics is the solution of linear ordinary and 
partial differential equations. As an undergraduate you were probably introduced to the 
method of separation of variables, which leads to a solution in terms of an eigenfunction 
expansion. However, this method is not the only one; there is Duhamel’s principle which 
uses the superposition integral. Here we expand upon this idea and illustrate how a solution, 
called a Green’s function, to a differential equation forced by the Dirac delta function can 
be used in an integral representation of a solution when the forcing is arbitrary. 


5.1 WHAT IS A GREEN’S FUNCTION? 


The following examples taken from engineering show how Green’s functions naturally 
appear during the solution of initial-value and boundary-value problems. We also show that 
the solution u(x) can be expressed as an integral involving the Green’s function and f(z). 


Circuit theory 


In electrical engineering, one of the simplest electrical devices consists of a voltage 
source u(t) connected to a resistor with resistance R and an inductor with inductance L. 
See Figure 5.1.1. Denoting the current by i(t), the equation that governs this circuit is 


di ; 
Le + Ri = v(t). (5.1.1) 


Consider now the following experiment: With the circuit initially dead, we allow the 
voltage to suddenly become Vo/Ar during a very short duration Ar starting at t = T. 
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R 


+ 
v(t) ~) i(t) L 


Figure 5.1.1: The RL electrical circuit driven by the voltage v(t). 


Then, at t = T + Ar, we again turn off the voltage supply. Mathematically, for t > T + Ar, 
the circuit’s performance obeys the homogeneous differential equation: 


P 
Lo +Ri=0, t>T+Ar, (5.1.2) 
whose solution is 
ilt) =Ibe 4/2, t>T+Ar, (5.1.3) 


where Io is a constant and L/R is the time constant of the circuit. Because the voltage v(t) 
during T < t < T + Az is Vo/Ar, then 


THAT 
/ v(t) dt = Vo. (5.1.4) 


Therefore, over the interval r < t < T + Ar, Equation 5.1.1 can be integrated to yield 


THAT THAT THAT 
1j di + Rf i(t) dt =| u(t) dt, (5.1.5) 
or 
THAT 
Lẹi(r + Ar) —i(7)] + Rf i(t) dt = Vo. (5.1.6) 
If i(t) remains continuous as Ar becomes small, then 
THAT 
Rf i(t) dt = 0. (5.1.7) 
Finally, because 
i(r)=0 and = i(r +Ar) = Ie POTAE ve Tye“ P/E, (5.1.8) 


for small Av, Equation 5.1.6 reduces to 
Ve 
Loet =W}, o I= AE (5.1.9) 


Therefore, Equation 5.1.3 can be written as 


0, t< T 
i(t) = (5.1.10) 
Yer 8, r<t 
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T THAT t 


Figure 5.1.2: The current i(t) within an RL circuit when the voltage Vo/Ar is introduced between the 
times T < t< T +AT. 


after using Equation 5.1.9. Equation 5.1.10 is plotted in Figure 5.1.2. 

Consider now a new experiment with the same circuit where we subject the circuit 
to N voltage impulses, each of duration Av and amplitude V;/Ar with i = 0,1,..., N, 
occurring at t = 7;. See Figure 5.1.3. The current response is then 


0, t < To, 
Voe BET )/E JT, To <t<7, 
Voe Rlt—70)/L 7, + Yea ir, T <t<t, 
i() = (5.1.11) 
N 
z Veo wey TN <t< TN+1- 
i=0 


Finally, consider our circuit subjected to a continuous voltage source v(t). Over each 
successive interval dr, the step change in voltage is u(r) dr. Consequently, from Equation 
5.1.11 the response i(t) is now given by the superposition integral 


t t 
i(t) = i MT) 6 Reni dr, or i(t)= i v(t) g(t|r) dr, (5.1.12) 
where e- R(t=7)/L 
g(t|r) = a Te bs (5.1.13) 


Here we have assumed that i(t) = v(t) = 0 for t < 7. In Equation 5.1.13, g(t|T) is called the 
Green’s function. As this equation shows, given the Green’s function to Equation 5.1.1, the 
response i(t) to any voltage source u(t) can be obtained by convolving the voltage source 
with the Green’s function. 

We now show that we could have found the Green’s function, Equation 5.1.13, by 
solving Equation 5.1.1 subject to an impulse- or delta-forcing function. Mathematically, 
this corresponds to solving the following initial-value problem: 


dg 


pa 
dt 


+ Rg =<d(t-7), g(O|r) = 0. (5.1.14) 
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Figure 5.1.3: The current i(t) within an RL circuit when the voltage is changed at t = To, t = 71, and so 
forth. 


Taking the Laplace transform of Equation 5.1.14, we find that 


esT ec ie 
At ltr) = at), (5.1.15) 


G(s|r) 


where H(-) is the Heaviside step function. As our short derivation showed, the most direct 
route to finding a Green’s function is solving the differential equation when its forcing 
equals the impulse or delta function. This is the technique that we will use throughout this 
chapter. 


| Statics 


Consider a string of length L that is connected at both ends to supports and is subjected 
to a load (external force per unit length) of f(a). We wish to find the displacement u(x) of 
the string. If the load f(x) acts downward (negative direction), the displacement u(x) of 
the string is given by the differential equation: 


ZE = f(x), (5.1.16) 


where T denotes the uniform tensile force of the string. Because the string is stationary at 
both ends, the displacement u(x) satisfies the boundary conditions u(0) = u(L) = 0. 
Instead of directly solving for the displacement u(x) of the string subject to the load 
f(x), let us find the displacement that results from a load ô(x — £) concentrated at the point 
x = &. See Figure 5.1.4. For this load, the differential equation, Equation 5.1.16, becomes 


d?g 
subject to the boundary conditions g(0|€) = g(L|€) = 0. 

In Equation 5.1.17, g(a|€) denotes the displacement of the string when it is subjected 
to an impulse load at x = £. In line with our circuit theory example, it gives the Green’s 
function for our statics problem. Once found, the displacement u(x) of the string subject to 
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Figure 5.1.4: The response, commonly called a Green’s function, of a string fixed at both ends to a point 
load at x = €. 


any arbitrary load f(x) can be found by convolving the load f(x) with the Green’s function 
g(x|£) as we did earlier. 

Let us now find this Green’s function. At any point x Æ €, Equation 5.1.17 reduces to 
the homogeneous differential equation: 


dq 
—= = 1.1 
Te 0, (5 8) 
which has the solution 
_ Jax+bd, OA <6, 
g(x\§) = eae Peper. (5.1.19) 


Applying the boundary conditions, Equation 5.1.19, we find that 
gO) =a-0+b=b=0, and g(Ll§) =cL+d=0, or d=-cL. (5.1.20) 
Therefore, we can rewrite Equation 5.1.19 as 


ax, 0<a<6, 
g(x|€) = eee baa (5.1.21) 
where a and c are undetermined constants. 
At x = €, the displacement u(x) of the string must be continuous; otherwise, the string 
would be broken. Therefore, the Green’s function given by Equation 5.1.21 must also be 
continuous there. Thus, 


ag 


a£ = c(€—L), or ee 


(5.1.22) 


From Equation 5.1.13 the second derivative of g(x|£) must equal the impulse function. 
Therefore, the first derivative of g(a|€), obtained by integrating this equation, must be 
discontinuous by the amount 1/T or 
dg(é — 1 
jan (AEE) _ doe- ee) 


= 5.1.2 
e—>0 dx dx T’ ( 2) 


in which case dg(Et|€) dg(€-|€) 1 
g mS 
é a a (5.1.24) 
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where €* and €~ denote points lying just above or below €, respectively. Using Equation 
5.1.24, we find that 


dg(&- |E) _ dg(ET|€)_ ag 
i and m es Eei (5.1.25) 
Thus, Equation 5.1.25 leads to 
Beea ns y a (5.1.26) 
€-L T €-L LT 
and the Green’s function is 
g(x|§) = ay (> — L)z<, (5.1.27) 


where z< = min(z,é) and ry = earn To find the displacement u(x) subject to the 
load f(x), we proceed as we did in the previous example. The result of this analysis is 


n= f Odd- 4 


since € < x in the first integral and x < € in the second integral of Equation 5.1.28. 


| Integral Equations 


Consider the Sturm-Liouville problem 


xf 16 (€-L)dé, (5.1.28) 


y” +y = 0, y(0) = y(L) =0. (5.1.29) 


From its general theory, nontrivial solutions exist only if 


2,2 


nen . (NTT 
An = T2’ Yn(x) = sin( 22) , (5.1.30) 
where n = 1, 2,3,.... 
Consider now a new boundary-value problem: 
dy 
qe fe) y0) = y(L) = 0. (5.1.31) 


In the next section (Equation 5.2.76), we will show that we can write its solution by 


=f F(E)g(al€) d£, (5.1.32) 
where the Green’s function g(x|£) is given by 
d2 
T3 =el- E), (OE) = G(LIG)=0, or g(æl£) = (L- r>)z</L, (6.1.38) 


where z> = max(x,&) and z< = min(z, £). 
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We can now use Equation 5.1.29 to rewrite Equation 5.1.31 as Ay() = f(€). Multi- 
plying this equation by g(x|€) and integrating from 0 to L, we find that 


T f(©)g(al€) dé = af ve g(a) dé, (5.1.34) 
or 
a f v g(xl|ë) dé = 0. (5.1.35) 


Because of the equivalence of Equation 5.1.29 and Equation 5.1.35, the solutions to the 
integral equation, Equation 5.1.35, are An = n?n? /L? with yn(x) = sin(nrg/L). Direct 
substitution verifies this result. Thus, we can use Green’s functions to construct integral 
equations that have known solutions. Indeed, it was the use of Green’s functions to solve 
Fredholm integral equations that drew the attention of mathematicians at the turn of the 
twentieth century.! 


5.2 ORDINARY DIFFERENTIAL EQUATIONS 


Second-order differential equations are ubiquitous in engineering. In electrical engi- 
neering, many electrical circuits are governed by second-order, linear ordinary differential 
equations. In mechanical engineering they arise during the application of Newton’s second 
law. 

One of the drawbacks of solving ordinary differential equations with a forcing term is 
its lack of generality. Each new forcing function requires a repetition of the entire process. 
In this section we give some methods for finding the solution in a somewhat more gen- 
eral manner for stationary systems where the forcing, not any initially stored energy (i.e., 
nonzero initial conditions), produces the total output. Unfortunately, the solution must be 
written as an integral. 

Consider the linear differential equation 


y” +2y' +y = f(t), (5.2.1) 


subject to the initial conditions y(0) = y'(0) = 0. Solving this equation by Laplace trans- 
forms, we can write the Laplace transform of y(t), Y (s), as the product of two Laplace 


transforms: 1 


9) = Grip 


One drawback in using Equation 5.2.2 is its dependence upon an unspecified Laplace trans- 
form F(s). Is there a way to eliminate this dependence and yet retain the essence of the 
solution? 

One way of obtaining a quantity that is independent of the forcing is to consider the 
ratio: 


F(s). (5.2.2) 


LUR E E 


This ratio is called the transfer function because we can transfer the input F(s) into the 
output Y (s) by multiplying F(s) by G(s). It depends only upon the properties of the 
system. 


(5.2.3) 


1 See Section 36 in Kneser, A., 1911: Integralgleichungen und ihre Anwendungen in der mathematischen 
Physik. Braunschweig, 293 pp. 
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Let us now consider a problem related to Equation 5.2.1, namely 
g” +29 +g = ô(t), t>0, (5.2.4) 


with g(0) = g'(0) = 0. Because the forcing equals the Dirac delta function, g(t) is called 
the impulse response or Green’s function.? Computing G(s), 


1 


G9) = Gyr (5.2.5) 


From Equation 5.2.3 we see that G(s) is also the transfer function. Thus, an alternative 
method for computing the transfer function is to subject the system to impulse forcing and 
the Laplace transform of the response is the transfer function. 
From Equation 5.2.3, 
Y(s) = G(s) F(s), (5.2.6) 


or 
y(t) = g(t) * f(t). (5.2.7) 


That is, the convolution of the impulse response with the particular forcing gives the re- 
sponse of the system. Thus, we may describe a stationary system in one of two ways: (1) 
in the transform domain we have the transfer function, and (2) in the time domain there is 
the impulse response. 

Despite the fundamental importance of the impulse response or Green’s function for a 
given linear system, it is often quite difficult to determine, especially experimentally, and a 
more convenient practice is to deal with the response to the unit step H(t). This response 
is called the indicial admittance or step response, which we shall denote by a(t). Because 
L[H(t)] = 1/s, we can determine the transfer function from the indicial admittance because 
Lla(t)] = G(s)L[A(t)] or sA(s) = G(s). Furthermore, because 


LIO = Gls) = FY (5.28) 
then F 
g(t) = n (5.2.9) 


since L[f’(t)] = sF (s) — f(0*). 
e Example 5.2.1 
Let us find the transfer function, impulse response, and step response for the system 
y” — 3y' + 2y = f(t), (5.2.10) 
with y(0) = y'(0) = 0. To find the impulse response, we solve 


g” — 3g' + 2g = ô(t— 7T), (5.2.11) 


2 For the origin of the Green’s function, see Farina, J. E. G., 1976: The work and significance of George 
Green, the miller mathematician, 1793-1841. Bull. Inst. Math. Appl., 12, 98-105. 
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with g(0) = g'(0) = 0. We have generalized the problem to an arbitrary forcing at t = T 
and now denote the Green’s function by g(t|T). We have done this so that our discussion 
will be consistent with the other sections in the chapter. 

Taking the Laplace transform of Equation 5.2.11, we find that 


=ar 


e 


OILY EEE 


(5.2.12) 


which is the transfer function for this system when 7 = 0. The impulse response or Green’s 
function equals the inverse of G(s|7) or 


g(t|r) = [ext —e'-*| H(t—7). (5.2.13) 


To find the step response, we solve 
a” — 3a’ + 2a = H(t), (5.2.14) 


with a(0) = a’(0) = 0. Taking the Laplace transform of Equation 5.2.14, 


1 
A(s\ = = 5.2.15 
() = E (5.2.15) 
and the indicial admittance is given by the inverse of Equation 5.2.15, or 
a(t) =} +e” — et. (5.2.16) 


Note that a’(t) = g(t|0). 


e Example 5.2.2 


MATLAB’s control toolbox contains several routines for the numerical computation 
of impulse and step responses if the transfer function can be written as the ratio of two 
polynomials. To illustrate this capacity, let us redo the previous example where the transfer 
function is given by Equation 5.2.12 with r = 0. The transfer function is introduced by 
loading in the polynomial in the numerator num and in the denominator den followed by 
calling tf. The MATLAB script 


clear 

% load in coefficients of the numerator and denominator 
% of the transfer function 

num = [0 0 1]; den = [1 -3 2]; 

% create the transfer function 

sys = tf(num,den); 

% find the step response, a 

[a,t] = step(sys); 

% plot the indicial admittance 
subplot(2,1,1), plot(t, a, ’o’) 
ylabel(’indicial response’ ,’Fontsize’ ,20) 
% find the impulse response, g 

[g,t] = impulse(sys) ; 

% plot the impulse response 
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Figure 5.2.1: The impulse and step responses corresponding to the transfer function, Equation 5.2.12, 
with 7 = 0. 


subplot (2,1,2), plot(t, g, ’o’) 
ylabel(’impulse response’, ’Fontsize’ ,20) 
xlabel(’time’,’Fontsize’ ,20) 


shows how the impulse and step responses are found. Both of them are shown in Figure 
5.2.1. 


e Example 5.2.3 


There is an old joke about a man who took his car into a garage because of a terrible 
knocking sound. Upon his arrival the mechanic took one look at it and gave it a hefty 
kick.3 Then, without a moment’s hesitation he opened the hood, bent over, and tightened 
up a loose bolt. Turning to the owner, he said, “Your car is fine. That’ll be $50.” The 
owner felt that the charge was somewhat excessive, and demanded an itemized account. 
The mechanic said, “The kicking of the car and tightening one bolt, cost you a buck. The 
remaining $49 comes from knowing where to kick the car and finding the loose bolt.” 

Although the moral of the story may be about expertise as a marketable commodity, 
it also illustrates the concept of transfer function.* Let us model the car as a linear system 
where the equation 


d”y daty d 


an pn F On—1 dal teeta +aoy = f(t) (5.2.17) 


governs the response y(t) to a forcing f(t). Assuming that the car has been sitting still, the 
initial conditions are zero and the Laplace transform of Equation 5.2.17 is 


K(s)Y(s) = F(s), (5.2.18) 


where 
K(s) = ans” + Gn_18™ | +--+ + ais + ao. (5.2.19) 


3 This is obviously a very old joke. 


4 Originally suggested by Stern, M. D., 1987: Why the mechanic kicked the car - A teaching aid for 
transfer functions. Math. Gaz., 71, 62-64. 
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Hence, 


Y(s) = = G(s)F(s), (5.2.20) 


where the transfer function G(s) clearly depends only on the internal workings of the car. 
So if we know the transfer function, we understand how the car vibrates because 


ut) = f g(t — x) f(x) dx. (5.2.21) 


But what does this have to do with our mechanic? He realized that a short sharp kick 
mimics an impulse forcing with f(t) = 6(t) and y(t) = g(t). Therefore, by observing the 
response of the car to his kick, he diagnosed the loose bolt and fixed the car. 


In the previous examples, we used Laplace transforms to solve for the Green’s functions. 
However, there is a rich tradition of using Fourier transforms rather than Laplace transforms. 
In these particular cases, the Fourier transform of the Green’s function is called frequency 
response or steady-state transfer function of our system when 7 = 0. Consider the following 
examples. 


e Example 5.2.4: Spectrum of a damped harmonic oscillator 


In mechanics the damped oscillations of a mass m attached to a spring with a spring 
constant k and damped with a velocity-dependent resistance are governed by the equation 


my” + cy + ky = f(t), (5.2.22) 


where y(t) denotes the displacement of the oscillator from its equilibrium position, c denotes 
the damping coefficient, and f(t) denotes the forcing. 

Assuming that both f(t) and y(t) have Fourier transforms, let us analyze this system 
by finding its frequency response. We begin by solving for the Green’s function g(t|rT), 
which is given by 

mg” +cg' +kg = 5(t—7), (5.2.23) 


because the Green’s function is the response of a system to a delta function forcing. Taking 
the Fourier transform of both sides of Equation 5.2.23, the frequency response is 


eer ee" rn 
G = = 5.2.24 
elt) k+icew— mw? w + icw/m— w? ( ) 


where we = k/m is the natural frequency of the system. The most useful quantity to plot 
is the frequency response or 


|G(w|r)| = (5.2.25) 


= (5.2.26) 


In Figure 5.2.2 we plotted the frequency response as a function of c?/(km). Note that as the 
damping becomes larger, the sharp peak at w = wo essentially vanishes. As c?/(km) — 0, 
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Figure 5.2.2: The variation of the frequency response for a damped harmonic oscillator as a function of 
driving frequency w. See the text for the definition of the parameters. 


we obtain a very finely tuned response curve. Let us now find the Green’s function. From 
the definition of the inverse Fourier transform, 


1 lore) iwt 1 (e0) iwt 
mg(t|r) = = | E 7 dw = J i dw, (5.2.27) 


-œ W? — icw/m — we 2T J_ (w — w1)(w — we) 


Wie = Ł/ w — 7? + yi, (5.2.28) 


and y = c/(2m) > 0. We can evaluate Equation 5.2.27 by residues. Clearly the poles always 
lie in the upper half of the w-plane. Thus, if t < 7 in Equation 5.2.27 we can close the line 
integration along the real axis with a semicircle of infinite radius in the lower half of the 
w-plane by Jordan’s lemma. Because the integrand is analytic within the closed contour, 
g(t\r) = 0 for t < r. This is simply the causality condition,’ the impulse forcing being the 
cause of the excitation. Clearly, causality is closely connected with the analyticity of the 
frequency response in the lower half of the w-plane. 

If t > 7, we close the line integration along the real axis with a semicircle of infinite 
radius in the upper half of the w-plane and obtain 


where 


(5.2.29) 
e7 It-T) sin| triw = J 
2 —t jew guatr] = ( ) 0 y H(t T) 
Wy — w2 we — 7? 
(5.2.30) 


Let us now examine the damped harmonic oscillator by describing the migration of 
the poles w;2 in the complex w-plane as y increases from 0 to oo. See Figure 5.2.3. For 
y < wo (weak damping), the poles w;,2 are very near to the real axis, above the points 
tw, respectively. This corresponds to the narrow resonance band discussed earlier and 
we have an underdamped harmonic oscillator. As y increases from 0 to wo, the poles 


5 The principle stating that an event cannot precede its cause. 
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Figure 5.2.3: The migration of the poles of the frequency response of a damped harmonic oscillator as a 
function of y. 


approach the positive imaginary axis, moving along a semicircle of radius wo centered at 
the origin. They coalesce at the point iwo for y = wo, yielding repeated roots, and we have 
a critically damped oscillator. For y > wo, the poles move in opposite directions along 
the positive imaginary axis; one of them approaches the origin, while the other tends to 
ico as y + oo. The solution then has two purely decaying, overdamped solutions. During 
the early 1950s, a similar diagram was invented by Evans® where the movement of closed- 
loop poles is plotted for all values of a system parameter, usually the gain. This root-locus 
method is very popular in system control theory for two reasons. First, the investigator can 
easily determine the contribution of a particular closed-loop pole to the transient response. 
Second, he can determine the manner in which open-loop poles or zeros should be introduced 
or their location modified so that he will achieve a desired performance characteristic for 
his system. 


e Example 5.2.5: Low-frequency filter 


Consider the ordinary differential equation 
Ry! + a = f(t), (5.2.31) 
where R and C are real, positive constants. If y(t) denotes current, then Equation 5.2.31 
would be the equation that gives the voltage across a capacitor in an RC circuit. Let us find 
the frequency response and Green’s function for this system. We begin by writing Equation 
5.2.31 as 


Rg + 4 = 5(t—7), (5.2.32) 


where g(t|7) denotes the Green’s function. If the Fourier transform of g(t|T) is G(w|r), the 
frequency response G(w|r) is given by 
GIT) 


iwRG(w|r) + a a NE (5.2.33) 


6 Evans, W. R., 1948: Graphical analysis of control systems. Trans. AIEE, 67, 547-551; Evans, W. 
R., 1954: Control-System Dynamics. McGraw-Hill, 282 pp. 
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Figure 5.2.4: The variation of the frequency response, Equation 5.2.35, as a function of driving frequency 
w. See the text for the definition of the parameters. 


or 
eet CeT tT 
= = 2.34 
GWln)= RPO 14 wae (nes) 
and c c 
|G(w|r)| = (5.2.35) 


V1 +w?R?2C? [t+ w/w? 


where w, = 1/(RC) is an intrinsic constant of the system. In Figure 5.2.4 we plotted 
|G(w|r)| as a function of w. From this figure, we see that the response is largest for small 
w and decreases as w increases. 

This is an example of a low-frequency filter because relatively more signal passes through 
at lower frequencies than at higher frequencies. To understand this, let us drive the system 
with a forcing function that has the Fourier transform Fw). The response of the system 
will be G(w,0)F(w). Thus, that portion of the forcing function’s spectrum at the lower 
frequencies is relatively unaffected because |G(w,0)| is near unity. However, at higher 
frequencies where |G(w,0)| is smaller, the magnitude of the output is greatly reduced. 


e Example 5.2.6 


During his study of tumor growth, Adam’ found the particular solution to an ordinary 
differential equation which, in its simplest form, is 


We 2 SD ee |z|/L — 1, |z| < L, 
y —ay= { 0, lol > L, (5.2.36) 


by the method of Green’s functions. Let us retrace his steps and see how he did it. 
The first step is finding the Green’s function. We do this by solving 


g" —a’g = ô(£), (5.2.37) 


subject to the boundary conditions limj,)-,.. g(x) — 0. Taking the Fourier transform of 
Equation 5.2.37, we obtain 
1 


w2 + a2" 


G(w) = (5.2.38) 


7 Adam, J. A., 1986: A simplified mathematical model of tumor growth. Math. Biosci., 81, 229-244. 
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The function G(w) is the frequency response for our problem. Straightforward inversion 
yields the Green’s function 


ele 


2a 
Therefore, by the convolution integral, y(x) = g(a) * f(a), 


g(x) =- (5.2.39) 


L L 
ves foes) (\el/b—1) de= 5 f G-I (5.2.40) 


To evaluate Equation 5.2.40 we must consider four separate cases: —oo < x < —L, 
—L<xz<0,0<x< L,and L< x< oœ. Turning to the —o0 < x < —L case first, we have 


L 
ve) = zg f | 1- E/E) -9 ag (5.2.41) 
OF 0 ax L 
~ Ta is (1+ €/L)e 6 dé + al (1 — €/L)e-™ dé (5.2.42) 
S T (er +e — 2). (5.2.43) 


Similarly, for x > L, 


L 
y(z) = ae (1—|é|/L) e729 dé (5.2.44) 
ae 4 eT or L 
~ 2a se (1 + €/L) e% dé + Da f (1 — €/L) e™ dé (5.2.45) 
~ iL (eee Ss (5.2.46) 


On the other hand, for —L < x < 0, we find that 


x L 
v= f 0-lea eee- ag + 5 f (1 E/E) e a (5.2.47) 
-ar z ax 0 ax L 
= Sa f arune E f arepeta = | Q-6/)e dg 
(5.2.48) 
1 —aL ax 
= BL [e°” cosh(ax) + a(x +L) — e% ]. (5.2.49) 
Finally, for 0 < x < L, we have that 
x L 
v= fa —lel/tyere ag + f lezen a (5.2.50) 
—axr 0 =ar x ax L 
= Sn is (1+ €/L) e° dé + =f (1 — €/L) ee dé + <] (1 — E/L) e~° dE 
(5.2.51) 
1 —aL —ar 
= BL le cosh(axz) + a(L— z) — e7% ]. (5.2.52) 
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These results can be collapsed down into 


1 —aL i ala 
ylz) = -57 le cosh(az) + a(L — |x|) — e~2! J (5.2.53) 
if |x| < L, and 
=a 
FEN e aL —aL 
y(x) BE (e% +e" — 2) (5.2.54) 


if |z| > L. 


| Superposition integral 


So far we showed how the response of any system can be expressed in terms of its 
Green’s function and the arbitrary forcing. Can we also determine the response using the 
indicial admittance a(t)? 

Consider first a system that is dormant until a certain time t = T1. At that instant we 
subject the system to a forcing H(t — 71). Then the response will be zero if t < 7 and will 
equal the indicial admittance a(t — Tı) when t > Tı because the indicial admittance is the 
response of a system to the step function. Here t—7, is the time measured from the instant 
of change. 

Next, suppose that we now force the system with the value f(0) when t = 0 and hold 
that value until t = 7. We then abruptly change the forcing by an amount f(m) — f(0) 
to the value f(7) at the time 7, and hold it at that value until t = 72. Then we again 
abruptly change the forcing by an amount f(72) — f(71) at the time 72, and so forth (see 
Figure 5.2.5). From the linearity of the problem, the response after the instant t = T, equals 
the sum 


y(t) = f(O)a(t) + [f (T1) — FO)Ja(t — 71) + [f(r2) — f(ri)a(t — 72) 
+++++[f(t%m) — f(tm-1)]a(t — Tr). (5.2.55) 


If we write f(t,) — f(te-1) = Afk and Tk — Tk-1 = Ate, Equation 5.2.55 becomes 
y(t) = f(O)a(t) + a a(t — Aya Arp. (5.2.56) 
kel ATR 


Finally, proceeding to the limit as the number n of jumps becomes infinite, in such a manner 
that all jumps and intervals between successive jumps tend to zero, this sum has the limit 


y(t) = f(O)a(t) +f f'(r)a(t — T) dr. (5.2.57) 


Because the total response of the system equals the weighted sum (the weights being a(t)) 
of the forcing from the initial moment up to the time t, we refer to Equation 5.2.57 as 
the superposition integral, or Duhamel’s integral, named after the French mathematical 


8 Duhamel, J.-M.-C., 1833: Mémoire sur la méthode générale relative au mouvement de la chaleur dans 


les corps solides plongés dans des milieux dont la température varie avec le temps. J. Ecole Polytech., 22, 
20-77. 
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4 
S(t) 


Figure 5.2.5: Diagram used in the derivation of Duhamel’s integral. 
physicist Jean-Marie-Constant Duhamel (1797-1872), who first derived it in conjunction 
with heat conduction. 


We can also express Equation 5.2.57 in several different forms. Integration by parts 
yields 


y(t) = O+ [Frye (t=T) dr = — |f ro a(t —T) : (5.2.58) 


e Example 5.2.7 
Suppose that a system has the step response of a(t) = A[1 — e~‘/7], where A and T 
are positive constants. Let us find the response if we force this system by f(t) = kt, where 


k is a constant. 
From the superposition integral, Equation 5.2.57, 


y(t) =0+ i kAļ1 — e7 &-7)/7) dr = kA[t — T — e™/T)]. (5.2.59) 


| Boundary-value problem 


One of the purposes of this book is the solution of a wide class of nonhomogeneous 
ordinary differential equations of the form 


= [pmg|+s@v=-1@), acess (5.2.60) 
with 
ayla) +agy'(a)=0, By y(b) + Boy’(b) = 0. (5.2.61) 


This is an example of a Sturm-Liouville-like equation 


a A + [q(x) +Ar(x)Jy=—f(z),  a<z<b, (5.2.62) 
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where À is a parameter. Here we wish to develop the Green’s function for this class of 
boundary-value problems. 
We begin by determining the Green’s function for the equation 


“ peo] + s(x)g = —ô(x — £), (5.2.63) 


subject to yet undetermined boundary conditions. We know that such a function exists for 
the special case p(x) = 1 and s(x) = 0, and we now show that this is almost always true 
in the general case. Presently we construct Green’s functions by requiring that they satisfy 
the following conditions: 


e g(x|£) satisfies the homogeneous equation f(x) = 0 except at x = £, 


e g(x|£) satisfies certain homogeneous conditions, and 


e g(z|€) is continuous at x = £. 


These homogeneous boundary conditions for a finite interval (a,b) will be 


aglaj) + azg'(al€)=0,  Bıg(blE) + Bog’ (b|E) = 0, (5.2.64) 


where g’ denotes the x derivative of g(a|€) and neither a nor b equals €. The coefficients 
qa, and a2 cannot both be zero; this also holds for 6; and 62. These conditions include the 
commonly encountered Dirichlet, Neumann, and Robin boundary conditions. 

What about the value of g'(x|£) at x = €? Because g(z|€) is a continuous function of 
x, Equation 5.2.63 dictates that there must be a discontinuity in g’(x|€) at x = €. We now 
show that this discontinuity consists of a jump in the value g’(x|) at x = €. To prove this, 
we begin by integrating Equation 5.2.63 from € — e to € + €, which yields 


dg(x|£) 


ple); 


E+E E+e 
+f s(x) g(a|€) dx = —1. (5.2.65) 


=e 


Because g(z|€) and s(x) are both continuous at x = £, 


Ete 
lim s(x)g(x|£) dx = 0. (5.2.66) 


e—>0 Ede 
Applying the limit « — 0 to Equation 5.2.65, we have that 


i a WED], 


(5.2.67) 


where ¿+ and €~ denote points just above and below x = €, respectively. Consequently, 
our last requirement on g(x|€) will be that 


e dg/dx must have a jump discontinuity of magnitude —1/p(£) at x = £. 
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Similar conditions hold for higher-order ordinary differential equations.’ 

Consider now the region a < x < €. Let y (x) be a nontrivial solution of the homo- 
geneous differential equation satisfying the boundary condition at x = a; then ayyi(a) + 
azy (a) = 0. Because g(x|€) must satisfy the same boundary condition, a, g(a) + a2’ (alg) 
= 0. Since the set a1, œz is nontrivial, then the Wronskian of yı and g must vanish at z = a 
or y1(a)g'(al€) — y;(a)g(alé) = 0. However, for a < x < £, both yı(x) and g(a|g) satisfy 
the same differential equation, the homogeneous one. Therefore, their Wronskian is zero 
at all points and g(z|€) = ciyi(a) for a < x < €, where cı is an arbitrary constant. In 
a similar manner, if the nontrivial function y2(x) satisfies the homogeneous equation and 
the boundary conditions at x = b, then g(#|€) = czy2(x) for € < x < b. The continuity 
condition of g and the jump discontinuity of g’ at x = € imply 


cayı (£) — c2y2(€) = 0, crys (E) — c2y2 (£) = 1/p(E). (5.2.68) 


We can solve Equation 5.2.68 for cı and c2 provided the Wronskian of yı and y2 does not 
vanish at x = €, or 


yr(E)yo(€) — yo(E)ui (E) # 0. (5.2.69) 


In other words, yı (x) must not be a multiple of y2(x). Is this always true? The answer is 
“generally yes.” If the homogeneous equation admits no nontrivial solutions satisfying both 
boundary conditions at the same time,!° then y;(x) and y2(a) must be linearly independent. 
On the other hand, if the homogeneous equation possesses a single solution, say yo(«), which 
also satisfies a1 yo(a) + azy (a) = 0 and By, y0(b) + G2y6(b) = 0, then yı (x) will be a multiple 
of yo(a) and so is yo(a). Then they are multiples of each other and their Wronskian vanishes. 
This would occur, for example, if the differential equation is a Sturm-Liouville equation, À 
equals the eigenvalue, and yo(x) is the corresponding eigenfunction. No Green’s function 
exists in this case. 


e Example 5.2.8 


Consider the problem of finding the Green’s function for g” +k?g = —ô(x— £), 0< £ < 
L, subject to the boundary conditions g(0|€) = g(L|€) = 0 with k 4 0. The corresponding 
homogeneous equation is y” + k?y = 0. Consequently, g(z|€) = c1yı (x) = cı sin(kx) for 
0 <x <£, while g(x|f) = coyo(x) = cosin[k(L — x)] for € <a < L. 

Let us compute the Wronskian. For our particular problem, 


W(x) = yr(x)yo() — y1 (x)y2(x) (5.2.70) 
= —ksin(kx) cos[k(L — x)] — k cos(kx) sin[k(L — x)| (5.2.71) 
= —ksin|[k(a + L — x)| = —ksin(kL), (5.2.72) 


and W(€) = —ksin(kL). Therefore, the Green’s function will exist as long as kL 4 nr. If 
kL = nr, yi(x) and yo(x) are linearly dependent with yo(x) = c3 sin(n7ta/L), the solution 
to the regular Sturm-Liouville problem y” + Ay = 0, and y(0) = y(L) = 0. 


9 Ince, E. L., 1956: Ordinary Differential Equations. Dover Publications, Inc. See Section 11.1. 


10 In the theory of differential equations, this system would be called incompatible: one that admits no 
solution, save y = 0, which is also continuous for all x in the interval (a,b) and satisfies the homogeneous 
boundary conditions. 
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L 9(x/§) 


Figure 5.2.6: The Green’s function, Equation 5.2.75, divided by L, as functions of x and é when kL = 10. 


Let us now proceed to find g(z|€) when it does exist. The system, Equation 5.2.68, has 
the unique solution 


y2(£) _ Vi (£) 
awe ™ e Owe PRR 
where W (£) is the Wronskian of yı (x) and yọ(x) at x = €. Therefore, 
yı(z<)y2(z>) 
g(x\§) = — OWE (5.2.74) 


Clearly g(a|€) is symmetric in x and €. It is also unique. The proof of the uniqueness is as 
follows: We can always choose a different yı (x), but it will be a multiple of the “old” y1 (2), 
and the Wronskian will be multiplied by the same factor, leaving g(x|€) the same. This is 
also true if we modify y2(x) in a similar manner. 


e Example 5.2.9 


Let us find the Green’s function for g” + k?g = —d(x — £), 0 < x < L, subject to 
the boundary conditions g(0|€) = g(Z|€) = 0. As we showed in the previous example, 
yi(z) = c sin(ka), yo(a) = cosin[k(L — x)|, and W(é) = —ksin(kL). Substituting into 
Equation 5.2.74, we have that 


_ sin(ka<) sinfk(L — x>)] 
g(a) = k sin(kL) 


(5.2.75) 


where z< = min(z,&) and x, = max(z,¢). Figure 5.2.6 illustrates Equation 5.2.75. 


So far, we showed that the Green’s function for Equation 5.2.63 exists, is symmetric, 
and enjoys certain properties (see the material in the boxes after Equation 5.2.63 and 
Equation 5.2.67). But how does this help us solve Equation 5.2.63? We now prove that 


b 
y(a) = f alle) F(E) d£ (5.2.76) 
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is the solution to the nonhomogeneous differential equation, Equation 5.2.63, and the ho- 
mogeneous boundary conditions, Equation 5.2.64. 

We begin by noting that in Equation 5.2.76 x is a parameter while € is the dummy 
variable. As we perform the integration, we must switch from the form for g(a|€) for £ < x 
to the second form for € > x when € equals x; thus, 


x b 
ya) = | alate rede f IONE a. (5.2.77) 
Differentiation yields 
£ f soroas [PE feat + olele), (5.2.78) 
and 
= SE (zlé) f(E) dé = a VED ed- glelet) f(e). (5.2.79) 
Because g(x|£) is continuous everywhere, we have that g(x|xt) = g(x|x7) so that 
ow J= dg Coy £) dé 4 ee (218) ee) ae. (5.2.80) 


Differentiating once more gives 


ay Palas) ¢ dg(x|a~) gla Is) dg(x\x*) 
Py L f POI peo ae WEED play f PAIO so) ae WEED fe), (5.2.81 
The second and fourth terms on the right side of Equation 5.2.81 will not cancel in this 


case; on the contrary, 


dg(z|z~)  dg(z|z*) 1 
ie ae (5.2.82) 


To show this, we note that the term dg(x|x~)/dx denotes a differentiation of g(a|€) with 
respect to x using the x > € form and then letting é —> x. Thus, 


dg(cle~) y, BONE _ _ whe) (o) 


du FOON OLO age) 

while for dg(x|x+)/dax we use the x < € form or 
dglelet) _ 5, EE) nemla) 

da TE OWE) PAWE) Basy 
Upon introducing these results into the differential equation 

(a) 4 tle) Z + sley = -Fl@), (5.2.85) 
we have 
[ WEEE) +o eg! + slale FE a (5.2.86) 

b 
+ S WOED HEE + IENE d - a) = — F(a, 


238 Advanced Engineering Mathematics: A Second Course 


Because 
p(x)g" (x|€) + p(x) 9’ (z€) + s(x) g(x|€) = 0, (5.2.87) 


except for x = €, Equation 5.2.86, and thus Equation 5.2.63, is satisfied. Although Equation 
5.2.87 does not hold at the point x = £, the results are still valid because that one point 
does not affect the values of the integrals. As for the boundary conditions, 


b b 
TE / oraw (a) = if WAS pe) ae, (5.2.88) 


and a y(a) + azy' (a) = 0 from Equation 5.2.64. A similar proof holds for x = b. 
Finally, let us consider the solution for the nonhomogeneous boundary conditions 
ayla) + azy' (a) = a, and 6, y(b) + boy’ (b) = 8. The solution in this case is 


_ ale) Bw) a 
cam a1Yy2(a) + a2ys(a) Biyr(b) + Bays (b) | J (El FE) dE- eae) 


A quick check shows that Equation 5.2.89 satisfies the differential equation and both non- 
homogeneous boundary conditions. 


| Eigenfunction expansion 


We just showed how Green’s functions can be used to solve the nonhomogeneous linear 
differential equation. The next question is how do you find the Green’s function? Here we 
present the most common method: series expansion. This is not surprising given its success 
in solving the Sturm-Liouville problem. 

Consider the nonhomogeneous problem 


y” =- f(x), with y(0)=y(L)=0. (5.2.90) 
The Green’s function g(x|£) must therefore satisfy 


g" =—8d(e—&), with g(0|€) = g(LIé) =0. (5.2.91) 


Because g(z|€) vanishes at the ends of the interval (0, L), this suggests that it can be 
expanded in a series of suitably chosen orthogonal functions such as, for instance, the 


Fourier sine series 
[0.0] 


g(x|€) = LG (é)sin(“"*),, (5.2.92) 


where the expansion coefficients Gn are dependent on the parameter €. Although we chose 
the orthogonal set of functions sin(n7a/L), we could have used other orthogonal functions 
as long as they vanish at the endpoints. 
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Because Bs as 
ile) = (-) Gat sin( 7), (5.2.98) 
n=1 
and x 
lx- £) = 2 An (Ê) sin(“**) 5 (5.2.94) 
where E: , 
An(§) = d 6(a — £) sin( 25) dz = zsin( 7) (5.2.95) 


z 2 (E) Gal€)sin( F) = -2 2 sin( 27) sin( 5), (5.2.96) 


after substituting Equation 5.2.93 through Equation 5.2.95 into the differential equation, 
Equation 5.2.91. Since Equation 5.2.96 must hold for any arbitrary zx, 


nên? 2 nm& 
Fz n = —sin{ —— }. 5.2.97 
(SF) a zsa( “7 ( ) 
Thus, the Green’s function is 
2b 1 
gal) = 2 a sin( sin( Z2) . (5.2.98) 


How might we use Equation 5.2.98? We can use this series to construct the solution of 
the nonhomogeneous equation, Equation 5.2.90, via the formula 


vo = f ol Nd (5.2.99) 
This leads to PEN i 
ye) = 5 ae sin(™) | f(e)sin( 7) dé, (5.2.100) 
7 ne 
ylz) = = > Sr in( 22) , (5.2.101) 
where a,, are the Fourier sine coefficients of f(x). 
e Example 5.2.10 
Consider now the more complicated boundary-value problem 
y!" +k’y=—f(z), with y(0)=y(L)=0. (5.2.102) 


The Green’s function g(z|€) must now satisfy 


g! +kg =-d(a—£), and g(0|é) = g(LI€) =0. (5.2.103) 


240 Advanced Engineering Mathematics: A Second Course 


Once again, we use the Fourier sine expansion 


Co 


g(x|€) = Len Es). (5.2.104) 


Direct substitution of Equation 5.2.104 and Equation 5.2.94 into Equation 5.2.103 and 
grouping by corresponding harmonics yields 


n?n? 2 2 nrTÊ 
ZL asn Sy, 5.2.10 
Tr Gale) +G = —Zsin( 22E) (5.2.105) 
k 2 _sin(nné/L) 
sin(nr 
Thus, the Green’s function is 
X sin(nrë/L)sin e 
g(a|é) = 2 3 ree ( (5.2.107) 


Examining Equation 5.2.107 more closely, we note that it enjoys the symmetry property 
that g(a|g) = g(é|2). 


e Example 5.2.11 


Let us find the series expansion for the Green’s function for 


2 
ag’ +g 4 (ite ~ Jo =-d(4-—£), 0<a<L, (5.2.108) 


where m > 0 and is an integer. The boundary conditions are 
lim \g(al€)|< oo, and g(L\€) =0. (5.2.109) 
2S 


To find this series, consider the Fourier-Bessel series 


9(2l€) = XC Gnl) Jm(knm2), (5.2.110) 


3 
Il 
m 


where knm is the nth root of Jm(knmL) = 0. This series enjoys the advantage that it satisfies 
the boundary conditions and we will not have to introduce any homogeneous solutions so 
that g(x|£) satisfies the boundary conditions. 

Substituting Equation 5.2.110 into Equation 5.2.108 after we divide by x and using the 
Fourier-Bessel expansion for the delta function, we have that 


(k? — kam)Gn(£) = Tc en DË T TU Bom)’ (5.2.111) 


so that 


(elf) =F ple -~ =a aaa (5.2.112) 
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L g(x/§) 


Figure 5.2.7: The Green’s function, Equation 5.2.112, as functions of x/L and €/L when kL = 10 and 
m=1. 


Equation 5.2.112 is plotted in Figure 5.2.7. 


We summarize the expansion technique as follows: Suppose that we want to solve the 
differential equation 
Ly(x) = —f(2), (5.2.113) 


with some condition By(x) = 0 along the boundary, where L now denotes the Sturm- 
Liouville differential operator 


i= £ KA + [q(x) + Ar(z)], (5.2.114) 


and B is the boundary condition operator 


d 
ay tary at x =a, 
B= 5 (5.2.115) 


d 
bı + Poa—, at x = b. 
dx 
We begin by seeking a Green’s function g(z|€), which satisfies 
Lg = —0(a — £), Bg =0. (5.2.116) 


To find the Green’s function, we utilize the set of eigenfunctions y,,(x) associated with the 
regular Sturm-Liouville problem 


d 


a. (a) | + (a(x) + Ant(z)]~n = 0, (5.2.117) 


dx 
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where Y,,(«) satisfies the same boundary conditions as y(x). If g exists and if the set {y,} 
is complete, then g(a|€) can be represented by the series 


g(2lé) = X` Gn (Eon (2). (5.2.118) 
n=1 
Applying L to Equation 5.2.118, 


Lg(a|€) = 


Me 


Gn(€)Llyn(2)] = X Gn(E)(A— An)r(@)pn(z) = —6(e@-€),  (5.2.119) 


Il 
= 


n 


if A does not equal any of the eigenvalues An. Multiplying both sides of Equation 5.2.119 
by %m(a) and integrating over x, 


oo b 
SOCON- An) fra) en()em(2) dz =~ m6) (5.2.120) 
n=1 Q 
If the eigenfunctions are orthonormal, 
b 
aoe 1, n= m, TER Pn lE) 
ie r(£)Pn(L)~m(ax) dx = T A. and G,(&) = ay (5.2.121) 
This leads directly to the bilinear formula: 
> Pn (E) Pn(T 
g(zlé) =Y 2a (5.2.122) 


n=1 


which permits us to write the Green’s function at once if the eigenvalues and eigenfunctions 
of L are known. 


Problems 
For the following initial-value problems, find the transfer function, impulse response, Green’s 
function, and step response. Assume that all of the necessary initial conditions are zero and 


T > 0. If you have MATLAB’s control toolbox, use MATLAB to check your work. 


1. g + kg = 6(t— 7) 2. g” —2g —3g=d(t—T) 3. g” +4g' +3g = O(t—7) 


4. g" —2g' +5g=d(t-7) 5. g”—3g' +2 =d(t—7) 6. 9" +49’ + 4g = d(t - 7) 
7. g" — 9g =6(t—7) 8. g" +g = ô(t — 7) 9. g” — g' = d(t 7) 


Find the Green’s function and the corresponding bilinear expansion using eigenfunctions 


from the regular Sturm-Liouville problem yi’ + kžyn = 0 for 


g” = —ô(x — £), 0< x, < L, 
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which satisfy the following boundary conditions: 


10. g(0|€) — ag’ (OIE) =0,a#40,-L, g (LIE) =0, 
11. g(O|) — g'(O|g) = 9, A LI§) — g'(L|£) = 9, 
12. g(O|€) — g'(0|£) = 0, g(LIE) + g'(LIg) = 0. 


Find the Green’s function!! and the corresponding bilinear expansion using eigenfunctions 
from the regular Sturm-Liouville problem yi’ + k?.p, = 0 for 


g” — k?°g = —ô(x — £), 0<x,é< L, 


which satisfy the following boundary conditions: 


13. g(0|£) = 0, g(L|€) = 0, 
14. g'(0|£) = 0, g' (Lg) = 0, 
15. g(0|f) = 0, g(L|€) + g'(LI§) = 0, 
16. g(O|§) = 0, g(LI§) — 9 (LIE) = 0, 
17. ag(O|§) + g'(0|£) = 0, g' (Lg) = 0, 
18. g(0|f) + g'(0|£) = 0, g(L|§) — g' (Lg) = 0. 


5.3 JOINT TRANSFORM METHOD 


In the previous section an important method for finding Green’s function involved either 
Laplace or Fourier transforms. In the following sections we wish to find Green’s functions 
for partial differential equations. Again, transform methods play an important role. We will 
always use the Laplace transform to eliminate the temporal dependence. However, for the 
spatial dimension we will use either a Fourier series or Fourier transform. Our choice will 
be dictated by the domain: If it reaches to infinity, then we will employ Fourier transforms. 
On the other hand, a domain of finite length calls for an eigenfunction expansion. The 
following two examples illustrate our solution technique for domains of infinite and finite 
extent. 


e Example 5.3.1: One-dimensional Klein-Gordon equation 


The Klein-Gordon equation is a form of the wave equation that arose in particle physics 
as the relativistic scalar wave equation describing particles with nonzero rest mass. In this 
example, we find its Green’s function when there is only one spatial dimension: 


82g 1 «(07g 
a Z (Sites) = —ô(x — £)d(t — T), (5.3.1) 


11 Problem 18 was used by Chakrabarti, A., and T. Sahoo, 1996: Reflection of water waves by a nearly 
vertical porous wall. J. Austral. Math. Soc., Ser. B, 37, 417-429. 
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where —oo < x, < co, 0 < t,7, cis a real, positive constant (the wave speed), and a is a 
real, nonnegative constant. The corresponding boundary conditions are 


| a g(x, tl€, 7) — 0, (5.3.2) 
200 
and the initial conditions are 

g(x, O|€, 7) = g(x, O|€,7) = 0. (5.3.3) 


We begin by taking the Laplace transform of Equation 5.3.1 and find that 


dG s+? ae 
qe ( z )c= lx — Ee". (5.3.4) 


C 


Applying Fourier transforms to Equation 5.3.4, we obtain 


2 oo ik(x—€) 2 OO dine M 
G(x, s\é,7) = ser | dk =< aa Sos et 6) aie. is am 
0 


Qqr -œ S$? +a? + he? T s2 +a? +k? 


Inverting the Laplace transform and employing the second shifting theorem, 


2 T sin[(t = Tt) Va? + kc? | cos|[k(x — €)] dk. (5.3.6) 
0 


g(x, t]€, T) = -H(t T) Va? + k2c2 


Equation 5.3.6 represents a superposition of homogeneous solutions (normal modes) to 
Equation 5.3.1. An intriguing aspect of Equation 5.3.6 is that this solution occurs every- 
where after t > 7. If |x — ¿| > e(t — T), these wave solutions destructively interfere so that 
we have zero there while they constructively interfere at those times and places where the 
physical waves are present. 

Applying integral tables to Equation 5.3.6, the final result is 


Cc 


g(2, tlg, T) = 5 Jo lave — 7)? — (2 = €)?/c? | Heft- 7) — |x — S|]. (5.3.7) 


Figure 5.3.1 illustrates this Green’s function. Thus, the Green’s function for the Klein- 
Gordon equation yields waves that propagate to the right and left from « = 0 with the 
wave front located at x = +ct. At a given point, after the passage of the wave front, the 
solution vibrates with an ever-decreasing amplitude and at a frequency that approaches a, 
the so-called cutoff frequency, at t —> oo. 

Why is a called a cutoff frequency? From Equation 5.3.5, we see that, although the 
spectral representation includes all of the wavenumbers k running from —co to oo, the 
frequency w = Vc?k? + a? is restricted to the range w > a from Equation 5.3.6. Thus, a is 
the lowest possible frequency that a wave solution to the Klein-Gordon equation may have 
for a real value of k. 
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9(x, tf6,t)/e 


10 . 5 a(x—)/c 


Figure 5.3.1: The free-space Green’s function g(x, t|€,7)/c for the one-dimensional Klein-Gordon equation 
at different distances a(x — €)/c and times a(t — T). 


e Example 5.3.2: One-dimensional wave equation on the interval 0 < x < L 


One of the classic problems of mathematical physics involves finding the displacement 


of a taut string between two supports when an external force is applied. The governing 
equation is 

Ou u 

be” gz 7 ft), 0<a<L, 0<t, (5.3.8) 


where c is the constant phase speed. 


In this example, we find the Green’s function for this problem by considering the 
following problem: 


2 
ge © aga 7 Ole OI- 7); 0<2,E<L, 0<t,7, (5.3.9) 
with the boundary conditions 


a1g(0, t|§,7) + Pige(0,t|€,7) =0, 0<t, (5.3.10) 
and 
a2g(L,tlE,7) + Boge(L,tl€,7) =0, 0<t, (5.3.11) 
and the initial conditions 
g(x, O|€,7) = g(x, 0,7) =0, O<a<L. (5.3.12) 
We start by taking the Laplace transform of Equation 5.3.9 and find that 


d2 2 _ 
n E = oe 5 E) er, 0<a<L, (5.3.13) 


with 
a1G(0, s|f,7) + 61G"(0, s|€,7) = 0, (5.3.14) 
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and 
a2G(L, s|€,7) + 2G" (L, s|£, T) = 0. (5.3.15) 


Problems similar to Equation 5.3.13 through Equation 5.3.15 were considered in the previous 
section. There, solutions were developed in terms of an eigenfunction expansion. Applying 
the same technique here, 


Lar Wo YnlE)pn(T 
Gaes mA, (5.3.16) 
n=1 n 


where n(x) is the nth orthonormal eigenfunction to the regular Sturm-Liouville problem 
p (x) + k?y(a) = 0, 0<a<L, (5.3.17) 

subject to the boundary conditions 
ai y(0) + Big’ (0) = 0, (5.3.18) 


and 


azp(L) + Bay! (L) = 0. (5.3.19) 


Taking the inverse of Equation 5.3.16, we have that the Green’s function is 


sles > onlen (a) Ect —7)] | iee (5.3.20) 


kne 


Let us illustrate our results to find the Green’s function for 


a c a = 5(x — £)5(t — 7), (5.3.21) 
with the boundary conditions 
g(0,tl€,7) = g(L, tl, T) = 0, 0<t, (5.3.22) 
and the initial conditions 
g(x, 0|E, T) = g(x, 0lé,7)'=0, Oar Ti (5.3.23) 


For this example, the Sturm-Liouville problem is 
p" (x) + k’y(x) = 0, O0<a<L, (5.3.24) 


with the boundary conditions y(0) = (L) = 0. The nth orthonormal eigenfunction for 
this problem is 


Pal) =f F sin(“**) (5.3.25) 
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c Q(x, t,t) 


Figure 5.3.2: The Green’s function cg(z, t|§,7) given by Equation 5.3.26 for the one-dimensional wave 
equation over the interval 0 < x < L as a function of location x/L and time c(t — T)/L with €/L = 0.2. 
The boundary conditions are g(0,t|€,7) = g(L,t|€,7) = 0. 

Consequently, from Equation 5.3.20, the Green’s function is 


g(x, tlé,7) = = [5 L sin (276) sin( 2) satn] ) H(t—7r). (5.3.26) 


n=1 


See Figure 5.3.2. 
5.4 WAVE EQUATION 


In Section 5.2, we showed how Green’s functions could be used to solve initial- and 
boundary-value problems involving ordinary differential equations. When we approach par- 
tial differential equations, similar considerations hold, although the complexity increases. 
In the next three sections, we work through the classic groupings of the wave, heat, and 
Helmholtz’s equations in one spatial dimension. All of these results can be generalized to 
three dimensions. 

Of these three groups, we start with the wave equation 

ou 18u 

Ox? eor 
where t denotes time, x is the position, c is the phase velocity of the wave, and q(x, t) is 
the source density. In addition to Equation 5.4.1 it is necessary to state boundary and 
initial conditions to obtain a unique solution. The condition on the boundary can be either 
Dirichlet or Neumann or a linear combination of both (Robin condition). The conditions 
in time must be Cauchy, that is, we must specify the value of u(x,t) and its time derivative 
at t = to for each point of the region under consideration. 

We begin by proving that we can express the solution to Equation 5.4.1 in terms of 
boundary conditions, initial conditions, and the Green’s function, which is found by solving 


3g 18g 
ðr? eot? 


= —q(x,t), (5.4.1) 


= —0(x — €)d(t — 7), (5.4.2) 
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where € denotes the position of a source that is excited at t = 7. Equation 5.4.2 expresses 
the effect of an impulse as it propagates from x = € as time increases from t = T. For 
t <7, causality requires that g(x, t|, T) = g:(x,t|€,7) = 0 if the impulse is the sole source 
of the disturbance. We also require that g satisfies the homogeneous form of the boundary 
condition satisfied by u. 

Our derivation starts with the equations 


Omer) LO alr). -, 


O&? C2 Or2 al, T), (5.4.3) 


and 
0?g(x,tl,7) 1 g(x, t]E,7) _ 
o£? 2 Or? E 
where we obtain Equation 5.4.4 from a combination of Equation 5.4.2 plus reciprocity, 
namely g(x, t|,7) = g(€,—rT|x,—t). Next we multiply Equation 5.4.3 by g(x, t|€,7) and 
Equation 5.4.4 by u(€,7) and subtract. Integrating over € from a to b, where a and b are 
the endpoints of the spatial domain, and over T from 0 to tt, where tt denotes a time 
slightly later than t so that we avoid ending the integration exactly at the peak of the delta 
function, we obtain 


tt b 3u z 9 é 
| f fote.) G7) ulg, T) noe ) 
0 a 


5(x — £)6(t — 7), (5.4.4) 


O&? 
Br om ate 2ul€,7 
+ 5 he n OED) = sælen ED] \ dé dr 
tt pb 
=u(a,t)~ | / a(&,7) g(a, t|, 7) dé dr. (5.4.5) 
Because 
ul, T 07 g(x, tl, 
seten Sa” ni u(€,T) co T) 
ð ðu 0 Og(x T 
= È [oet £n] - Fue, jaa + 2, (5.4.6) 
and 
ə? T o g(x T 
jeden A _ ag, 7) e 
o0 ðulE, T) ð ðglx, tl, T) 
= Zoete) a7 = Fug.) A) ; (5.4.7) 
we find that 
a Ou(E, T Og(x r) ] E= 
faete = ) ul, T) gl = i fl? 


i Og(x dule, r) 17 
sif ent EED O geen ED] ag 
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The integrand in the first integral is specified by the boundary conditions. In the second 
integral, the integrand vanishes at t = ¢* from the initial conditions on g(z,t|€,7). The 
limit at t = 0 is determined by the initial conditions. Hence, 


tt pb 
u(x,t) = f i E df dr 


E E a 2 Poetry) : 
Ef enno ye? en N a 


FiF? Og(a, t|€, 0) Ou(€, 0) 
_ af jug, 0) AE”) = g(x, t]€, 0) ð dé. (5.4.9) 


= 
Equation 5.4.9 gives the complete solution of the nonhomogeneous problem. The first two 
integrals on the right side of this equation represent the effect of the source and the boundary 
conditions, respectively. The last term involves the initial conditions; it can be interpreted 
as asking what sort of source is needed so that the function u(x,t) starts in the desired 
manner. 


e Example 5.4.1 


Let us apply the Green’s function technique to solve 

u Pu 

Ot? Ox?” 

subject to the boundary conditions u(0,t) = 0 and u(1,t) = t, O < t, and the initial 


conditions u(x,0) = x and u:(xz,0) =0,0<a< 1. 
Because there is no source term and c = 1, Equation 5.4.9 becomes 


0<a<1, 0<t, (5.4.10) 


u(x,t) -f [g(x,t|1, T)ue(1, T) — w(1, T)ge(x,t|1,T)] dr 
a [g(x, t|0, T)ue(0, 7) — u(0, T)ge(x, t|0, 7)] dr 


= | fu(E,0)gr (a, t|€, 0) — g(a, tlE, O)ue(E, 0)] d£. (5.4.11) 


Therefore we must first compute the Green’s function for this problem. However, we have 
already done this in Example 5.3.2 and it is given by Equation 5.3.26 with c = L = 1. 
Next, we note that g(a, t|1,7) = g(x, t|0,7) = 0 and u(0,T) = u-(€,0) = 0. Consequently, 
Equation 5.4.11 reduces to only two nonvanishing integrals: 


t 1 
u(x,t) = -f u(1, T)ge(x,t|1, 7) dr — Í u(€, 0)g,(a, t|&, 0) dé. (5.4.12) 
0 0 
If we now substitute for g(x, t|€,7) and reverse the order of integration and summation, 
t oo t 
J u(1,T)ge(x,t|1, 7) dr = 2 So(-1)" sin(n7x) f T sin[na(t — 7)| dr (5.4.13) 
0 n=l 0 


OO 


= 2t yp sin(n72) l sin[na(t — r)| d(t — 7) 


n=1 
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—2 Do "sin hs (t—T)sin[nx(t—7)|d(t—7) (5.4.14) 


f» u(1,7)ge(a, t]1,7) dr = -a51 ” sin(naa) Aa (5.4.15) 
0 
£35 xe waitara oes T)| ase) au — 7)] \ t 
0 
= 2 ss - sin(nmx) + = 3 =i sin(n7x) sin(nzt), 
= - (5.4.16) 
and 
f» u(£, 0)gr (x, t|E, 0) dé = -ay sin(n7x) cos(nent) | €sin(nm€) dé (5.4.17) 
=-—2 T sin(n7ax) cos(nrt) a 3 osre i (5.4.18) 
n=1 0 
= 2 3 =i sin(nrzg)cos(nrt). (5.4.19) 


sin(nrz)cos(nrt) 


2 Co 
+ 2 5 n2 sin(n7x) sin(nat). (5.4.20) 


The first summation in Equation 5.4.20 is the Fourier sine expansion for f(x) = x over the 
interval 0 < x < 1. Indeed, a quick check shows that the particular solution u,(x,t) = xt 
satisfies the partial differential equation and boundary conditions. The remaining two 
summations are necessary so that u(#,0) = x and w;(x,0) = 0. 


To apply Equation 5.4.9 to other problems, we must now find the Green’s function for 
a specific domain. In the following examples we illustrate how this is done using the joint 
transform method introduced in the previous section. Note that both examples given there 
were for the wave equation. 


e Example 5.4.2: One-dimensional wave equation in an unlimited domain 


The simplest possible example of Green’s functions for the wave equation is the one- 
dimensional vibrating string problem.!? In this problem the Green’s function is given by 
the equation 

ag 20°9 2 


12 See also Graff, K. F., 1991: Wave Motion in Elastic Solids. Dover Publications, Inc., Section 1.1.8. 
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where —co < x, < œ, and 0 < t,7. If the initial conditions equal zero, the Laplace 
transform of Equation 5.4.21 is 
ÊG 3 
dz? 


G =—d(x— Eje 5T, (5.4.22) 


where G(x, s|f,7) is the Laplace transform of g(x, t|€, T). To solve Equation 5.4.22 we take 
its Fourier transform and obtain the algebraic equation 


= exp(—ik€ — sT) 
Having found the joint Laplace-Fourier transform of g(x, t|€,7), we must work our way 
back to the Green’s function. From the definition of the Fourier transform, we have that 


est f>% etk(w—-§) 
Gla, slé,7) = = J pa E (5.4.24) 


To evaluate the Fourier-type integral, Equation 5.4.24, we apply the residue theorem. See 
Section 2.1. Performing the calculation, 


G(a, s|f, 7) = | PA = Se) (5.4.25) 


Finally, taking the inverse Laplace transform of Equation 5.4.25, 


c 
g(a, tle T) = zH @ -T — |e — €l/c), (5.4.26) 
or 
c 
glz, tig, T) = Hct — 7) + (x — §)] Hle(é— 7) — (2 — €)]. (5.4.27) 
We can use Equation 5.4.26 and the method of images to obtain the Green’s function 
for 
3g 18g 
2 ape 7 d(a — €)d(t —T), 0 < 2,t,€,7, (5.4.28) 


subject to the boundary condition g(0,t|€,7) = 0. 

We begin by noting that the free-space Green’s function,!® Equation 5.4.26, is the 
particular solution to Equation 5.4.28. Therefore, we need only find a homogeneous solution 
f(a, t|E, T) so that 


g(a, tlg, T) = silt =T= |æ = €l/c) + f(a, tl, 7) (5.4.29) 


13 In electromagnetic theory, a free-space Green’s function is the particular solution of the differential 
equation valid over a domain of infinite extent, where the Green’s function remains bounded as we approach 
infinity, or satisfies a radiation condition there. 
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9(X, tlE,t)/c 


Figure 5.4.1: The Green’s function g(x, t|€,7)/c given by Equation 5.4.30 for the one-dimensional wave 
equation for x > 0 at different distances x/€ and times c(t—T) subject to the boundary condition g(0, t|€,7) = 
0. 


satisfies the boundary condition at x = 0. 

To find f(x, t|, T), let us introduce a source at x = —€ at t = r. The corresponding 
free-space Green’s function is H(t — r — |x + €|/c). If, along the boundary x = 0 for any 
time t, this Green’s function destructively interferes with the free-space Green’s function 
associated with the source at x = €, then we have our solution. This will occur if our new 
source has a negative sign, resulting in the combined Green’s function 


olz, thé, r) = 5 [E-r — le- él/e) - H(t- r- e +€l/0)]. (5.4.30) 


See Figure 5.4.1. Because Equation 5.4.30 satisfies the boundary condition, we need no 
further sources. 


In a similar manner, we can use Equation 5.4.26 and the method of images to find the 
Green’s function for 

Oy 18g 

a2 age TIETO- 7); 0< x,t, £,T, (5.4.31) 


subject to the boundary condition gz(0, t|, T) = 0. 
We begin by examining the related problem 


g 18g 
ðr? 2 OF = d(x — €)d(t — T) + O(a +E- 7), (5.4.32) 


where —oo < x, < oo, and 0 < t,T. In this particular case, we have chosen an image that 
is the mirror reflection of ô(x — £). This was dictated by the fact that the Green’s function 
must be an even function of x along x = 0 for any time t. In line with this argument, 


g(a, tlé,7) = 5 [H(t—7 — |e — €l/e) + H(t- r- |e + €l/o)]. (5.4.33) 
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e Example 5.4.3: Equation of telegraphy 


When the vibrating string problem includes the effect of air resistance, Equation 5.4.21 


becomes 22 J 22 
I g 209 _ 2 = a: 
2 | 2y a l aTi lx — €)d(t — T), (5.4.34) 


where —oo < x, < oo, and 0 < t,7, with the boundary conditions 


| i g(a, té, T) —> 0 (5.4.35) 
x|— co 


and the initial conditions 
g(x, O|€,7) = g(x, O|€, 7) = 0. (5.4.36) 


Let us find the Green’s function. 
Our analysis begins by introducing an intermediate dependent variable w(z,t|é,7), 
where g(x, t|€,7) =e" w(z, t|, T). Substituting for g(x, t/€,7), we now have 


2 2 
a Pw- 228 = 25(0 Eilt- Te. (maar) 


Taking the Laplace transform of Equation 5.4.37, we obtain 


W s* — T—ST 
= ( 2 ) W = —ô(x — £)e™—5". (5.4.38) 


c 
Using Fourier transforms as in Example 5.3.1, the solution to Equation 5.4.38 is 
a EAEAN = 
Wie ae, pes ee eS SL (5.4.39) 
NEEG 


Employing tables to invert the Laplace transform and the second shifting theorem, we have 
that 


w(x, tlg T) = Se" h| yv- -@-EP/F| Het -7)- le- 6l, (5.4.40) 


g(a, té,7) = Set -y VEP (© — A | Het- 7) -leE | (8.4.41) 


Figure 5.4.2 illustrates Equation 5.4.41 when y = 1. 
e Example 5.4.4 


Let us solve!* the one-dimensional wave equation on an infinite domain: 


u Ou 


Ae eee cos(wt)d[a — X(t)], (5.4.42) 


14 See Knowles, J. K., 1968: Propagation of one-dimensional waves from a source in random motion. J. 
Acoust. Soc. Am., 43, 948-957. 
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9(x,t6,t)/e 
o 
w 


0 
0 10 (x-)/c 


Figure 5.4.2: The free-space Green’s function g(x, t|€, 7) /c for the one-dimensional equation of telegraphy 
with y = 1 at different distances (x — €)/c and times t — T. 


subject to the boundary conditions 


im u(x,t) 30, O<t, (5.4.43) 
zi} Co 
and initial conditions 

u(x, 0) = uz(x,0) = 0, —00 < T < œ. (5.4.44) 


Here w is a constant and X(t) is some function of time. 

With the given boundary and initial conditions, only the first integral in Equation 5.4.9 
does not vanish. Substituting the source term q(x, t) = cos(wt)d{a — X(t)] and the Green’s 
function given by Equation 5.4.26, we have that 


u(a,t) = f [. q(&,7)g(a, t|, 7) d€ dr (5.4.45) 
~ ; [ im cos(wr)d[§ — X(7)]| H(t — 7) — |x — £|) dé dr (5.4.46) 
= al Ht — T — |X (T) — z|] cos(wr) dr, (5.4.47) 
since c = 1. 
Problems 


1. By direct substitution, show! that 


g(x, t|0,0) = Jo(Vat )H (x) H(t) 


15 First proven by Picard, É., 1894: Sur une équation aux dérivées partielles de la théorie de la propa- 
gation de l’électricité. Bull. Soc. Math., 22, 2-8. 
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is the free-space Green’s function governed by 


3?g 1 
aroi + 49 = 9(2)5(4), —oo < x,t < co. 


2. Use Equation 5.3.20 to construct the Green’s function for the one-dimensional wave 
equation 
3g 8g 
Ot? = Ox? 
subject to the boundary conditions g(0,t|€,7) = gz(L,t|€,7) = 0, O < t, and the initial 
conditions that g(a, 0|€,7) = g(x, 0|§,7) =0,0<a< L. 


=0(a@—€E)d(t—T), O<a,E<L, 0<t,T, 


3. Use Equation 5.3.20 to construct the Green’s function for the one-dimensional wave 
equation 

3’g 8g 

Je pa T d(a—E)d(t-—T), O<a,E<L, 0<t,T, 


subject to the boundary conditions g,(0,¢|,7) = gz(L,t|€,7) = 0, 0 < t, and the initial 
conditions that g(x, 0|€,7) = g(x, 0|€,7) =0,0<a< L. 


4. Use the Green’s function given by Equation 5.3.26 to write down the solution to the 
wave equation Uz = Ugy on the interval 0 < x < L with the boundary conditions u(0,t) = 
u(L,t) = 0, 0 < t, and the initial conditions u(x,0) = cos(rx/L) and us(x,0) =0,0< x< 
L. 


5. Use the Green’s function given by Equation 5.3.26 to write down the solution to the wave 
equation Utt = Ugz on the interval 0 < x < L with the boundary conditions u(0, t) = e~* 
and u(L,t) = 0, 0 < t, and the initial conditions u(x,0) = sin(rxz/L) and u(z,0) = 1, 
O<a<L. 


6. Use the Green’s function that you found in Problem 2 to write down the solution to the 
wave equation Utt = Ure on the interval 0 < x < L with the boundary conditions u(0, t) = 0 
and u,(L,t) = 1, 0 < t, and the initial conditions u(#,0) = z and u(x,0)=1,0< x< L. 


7. Use the Green’s function that you found in Problem 3 to write down the solution to 
the wave equation Ut = Uz, on the interval 0 < x < L with the boundary conditions 
uz (0,t) = 1 and uz(L,t) = 0, 0 < t, and the initial conditions u(#,0) = 1 and uz(x,0) = 0, 
O<a<L. 


8. Find the Green’s function!® governed by 


Pg 3g g 
aR F2 Dy? = Ole E)d(t-—T), O<aE<L, 0<t,T, 


subject to the boundary conditions 


g,(0, tl€,7) = ga (L, t|, T) = 0, 0<t, 


16 Ozisik, M. N., and B. Vick, 1984: Propagation and reflection of thermal waves in a finite medium. 
Int. J. Heat Mass Transfer, 27, 1845-1854; Tang, D.-W., and N. Araki, 1996: Propagation of non- 
Fourier temperature wave in finite medium under laser-pulse heating (in Japanese). Nihon Kikai Gakkai 
Rombumshu (Trans. Japan Soc. Mech. Engrs.), Ser. B, 62, 1136-1141. 
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and the initial conditions 


g(x, O|€, 7) = g(x, O|€,7) = 0, O<ar<L. 


Step 1: If the Green’s function can be written as the Fourier half-range cosine series 
Die NTE 
gle, t16, 1) = +Go(tlr) + Ž L catir) cos( 2), 


so that it satisfies the boundary conditions, show that Gn (t|T) is governed by 


272 mE 


G! +2G', + = G,= cos( 27E) d(t—T), O<n. 


Step 2: Show that 
Go(t|r) = e7 sinh(t — r) H(t — 7), 


and 


G,(t|r) = cos( 27) eer) SinlPol§ =D rg _ 5), 1<21, 


n 


where Bn = \/(n7/L)? — 
Step 3: Combine the results from Steps 1 and 2 and show that 
g(a, t|é,r) = e7 sinh(t — 7) H(t — 7) /L 
+ 2e7— H(t —7)/L 


x Do al eos ME) cos(). 


n=1 


5.5 HEAT EQUATION 


In this section we present the Green’s function!” for the heat equation 
q 


ðu ou 


P oe q(x, t), (5.5.1) 


where t denotes time, x is the position, a? is the diffusivity, and q(x, t) is the source den- 
sity. In addition to Equation 5.5.1, boundary conditions must be specified to ensure the 
uniqueness of solution; the most common ones are Dirichlet, Neumann, and Robin (a linear 
combination of the first two). An initial condition u(x,t = to) is also needed. 

The heat equation differs in many ways from the wave equation and the Green’s function 
must, of course, manifest these differences. The most notable one is the asymmetry of the 


17 See also Carslaw, H. S., and J. C. Jaeger, 1959: Conduction of Heat in Solids. Clarendon Press, 
Chapter 14; Beck, J. V., K. D. Cole, A. Haji-Sheikh, and B. Litkouhi, 1992: Heat Conduction Using 
Green’s Functions. Hemisphere Publishing Corp., 523 pp.; Ozisik, M. N., 1993: Heat Conduction. John 
Wiley & Sons, Inc., Chapter 6. 
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heat equation with respect to time. This merely reflects the fact that the heat equation 
differentiates between past and future as entropy continually increases. 

We begin by proving that we can express the solution to Equation 5.5.1 in terms of 
boundary conditions, the initial condition, and the Green’s function, which is found by 
solving 

ðg 20°g 

Ot” Ax? 
where £ denotes the position of the source. From causality’® we know that g(x, t\€,7) = 0 
ift <7. We again require that the Green’s function g(x, t|€,7) satisfies the homogeneous 
form of the boundary condition on u(x,t). For example, if u satisfies a homogeneous or 
nonhomogeneous Dirichlet condition, then the Green’s function will satisfy the correspond- 
ing homogeneous Dirichlet condition. Although we will focus on the mathematical aspects 
of the problem, Equation 5.5.2 can be given the physical interpretation of the temperature 
distribution within a medium when a unit of heat is introduced at € at time T. 

We now establish that the solution to the nonhomogeneous heat equation can be ex- 
pressed in terms of the Green’s function, boundary conditions, and the initial condition. 
We begin with the equations 


a’ PulE,7) dul, 7) = 
o£? ðr 


= 0(a — €)d(t — 7), (5.5.2) 


q(£, T), (5.5.3) 


and 


8? g(x, t|, T) l Og(a,tl€,7) 
OE? l ar = —d(x — €)d(t— 7). (5.5.4) 


As we did in the previous section, we multiply Equation 5.5.3 by g(x, t|€,7) and Equation 
5.5.4 by u(€,7) and subtract. Integrating over € from a to b, where a and b are the endpoints 
of the spatial domain, and over T from 0 to t+, where t* denotes a time slightly later than 
t so that we avoid ending the integration exactly at the peak of the delta function, we find 


ef ae (£,7) a sæ nEn] dé dr 


af f ienen + see MED] xar 


=) k q(&, T)g(x, t|E, 7) d€ dr — u(x,t). (5.5.5) 


Applying Equation 5.4.6 and performing the time integration in the second integral, we 


finally obtain 
u(x,t) =) a (£, 7) g(a, t|E, 7) dé dr 


a dg(x eS 
+a f fene, sa Me ar 


b 
+f u(€, O)g(a, t]€, 0) dé, (5.5.6) 


18 The principle stating that an event cannot precede its cause. 
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where we used g(x, t|€,¢*) = 0. The first two terms in Equation 5.5.6 represent the familiar 
effects of volume sources and boundary conditions, while the third term includes the effects 
of the initial data. 


e Example 5.5.1: One-dimensional heat equation in an unlimited domain 


The Green’s function for the one-dimensional heat equation is governed by 


3g 20g 
at? aa? = d(a — €)d(t — 7), wo<2tE<o, 0<t,7, (5.5.7) 
subject to the boundary conditions limis g(a, t|, T) — 0, and the initial condition 
g(x, O|€,7) = 0. Let us find g(x, t|€, 7). 
We begin by taking the Laplace transform of Equation 5.5.7 and find that 
iG S ô(x 7 £) est 


G — 
dx? a? a? 


(5.5.8) 


Next, we take the Fourier transform of Equation 5.5.8 so that 
eik e—sT 


a? , 


(k? +b°)G(k, s|€,7) F (5.5.9) 


where G(k, s|€,7) is the Fourier transform of G(x, s|€,7) and b? = s/a?. 
To find G(x, s|€,7), we use the inversion integral 


e78T oo ei(@—)k 
Gla, slé7) = 5 J. rp i. (5.5.10) 


Transforming Equation 5.5.10 into a closed contour via Jordan’s lemma, we evaluate it by 
the residue theorem and find that 


& e7 ls—Elvs/a—sr EBEYI 
GTS = Spies (5.5.11) 
From a table of Laplace transforms we finally obtain 
H(t— 1) | s 
PUET) = ex ; 9, 
g(x, t|€, 7) Rega | aan) (5.5.12) 


after applying the second shifting theorem. 


The primary use of the fundamental or free-space Green’s function!® is as a particular 
solution to the Green’s function problem. For this reason, it is often called the fundamental 


19 In electromagnetic theory, a free-space Green’s function is the particular solution of the differential 
equation valid over a domain of infinite extent, where the Green’s function remains bounded as we approach 
infinity, or satisfies a radiation condition there. 
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Figure 5.5.1: The Green’s function, Equation 5.5.17, for the one-dimensional heat equation on the semi- 
infinite domain 0 < x < oo, and 0 < t—7, when the left boundary condition is g,(0,t|&,7) = 0 and 


€=0.5. 


heat conduction solution. Consequently, we usually must find a homogeneous solution so 
that the sum of the free-space Green’s function plus the homogeneous solution satisfies any 
boundary conditions. The following examples show some commonly employed techniques. 


e Example 5.5.2 


Let us find the Green’s function for the following problem: 

Og 30°9 

ðt | Aa? 

subject to the boundary conditions g(0,t|€,7) = 0, limz+.0 g(a, t|, 7) — 0, and the initial 
condition g(x, 0|, r) = 0. From the boundary condition g(0,t|€,7) = 0, we deduce that 
g(a, t|, T) must be an odd function in x over the open interval (—oo,0o). We find this 
Green’s function by introducing an image source of —ô(x + £) and resolving Equation 5.5.7 
with the source 6(a—€)d(t—7) —d(a+€)6(t—T). Because Equation 5.5.7 is linear, Equation 
5.5.12 gives the solution for each delta function and the Green’s function for Equation 5.5.13 


gate Et fag fof tO) wang 


= (x — €)d(t — 7), 0<2,E<co, 0<t,7, (5.5.13) 


4ra? (t — 7) 4a?(t— 7T) 4a? (t=T) 
H(t-7) r +E : zE 
Ta2(t — T) ai | 4a? (t — =| sinh sant Z =| . (5.5.15) 


In a similar manner, if the boundary condition at x = 0 changes to g,(0,t|€,7) = 0, 
then Equation 5.5.14 through Equation 5.5.15 become 


(ode 2 fep] Caa |+] TN (5.5.16) 


Ara? (t — T) 4a? (t— T) ~ da2(t — 7) 
H(t — T) ee + £2 ae 
Ta2(t — rT) exp fa cost | . (5.5.17) 


Figure 5.5.1 illustrates Equation 5.5.17 for the special case when € = 0.5. 
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e Example 5.5.3: One-dimensional heat equation on the interval 0 < x < L 


Here we find the Green’s function for the one-dimensional heat equation over the in- 
terval 0 < x < L associated with the problem 


ðu u 


where a? is the diffusivity constant. 
To find the Green’s function for this problem, consider the following problem: 


3g 8g 
Ot = Ox? 


= ô(x — €)d(t—7), 0<a,€<L, 0<t,7, (5.5.19) 
with the boundary conditions 
a1g(0, t|, T) ly Bigx(0, t|, 7) a 0, 0< t, (5.5.20) 


and 


a2g(L, tl, T) + Bage(L,t]€,7) =0, 0<t, (5.5.21) 


and the initial condition 
g(x, 0/€£,7) =0, O<a<L. (5.5.22) 


We begin by taking the Laplace transform of Equation 5.5.19 and find that 


G s læ- E) _,, 
g2 gzl a e, O<2#<b, (5.5.23) 
with 
a1G(0, s|€,7) + 81G" (0, s|, T) = 0, (5.5.24) 
and 
a2G(L, s|€, T) + B2G'(L, s|€,7) = 0. (5.5.25) 


Problems similar to Equation 5.5.23 through Equation 5.5.25 were considered in Section 
5.2. Applying this technique of eigenfunction expansions, we have that 


G(x, slé, 1) =e" an (5.5.26) 
n=1 n 


where yn(x) is the nth orthonormal eigenfunction to the regular Sturm-Liouville problem 
yp (x) + k’y(x) = 0, 0<a<L, (5.5.27) 
subject to the boundary conditions 


ai y(0) + Big’ (0) = 0, (5.5.28) 


and 


azp(L) + Bay! (L) = 0. (5.5.29) 
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Taking the inverse of Equation 5.5.26, we have that 


gla, tié, T) = |X vnlE)en (alee | H- 7). (5.5.30) 


n=1 


For example, let us find the Green’s function for the heat equation on a finite domain 


09 2 ag 
Ot Ox? 


=0(a—€)d(t—T), O<a,€E<L, 0<t,T, (5.5.31) 


with the boundary conditions g(0, t\,7) = g(L, t|€,7) = 0, 0 < t, and the initial condition 
g(x, 0|§,7) =0,0<a< L. 
The Sturm-Liouville problem is 


y" (x) +k’ y(x) = 0, 0<z< L, (5.5.32) 


with the boundary conditions y(0) = (L) = 0. The nth orthonormal eigenfunction to 


Equation 5.5.32 is 
2 
pale) =f F sin(“*)., (5.5.33) 


Substituting Equation 5.5.33 into Equation 5.5.30, we find that 


2 co 

g(@,t]€,7) = 5 ps sin( 27) sin(**) Seen H(t— r). (5.5.34) 
n=1 

On the other hand, the Green’s function for the heat equation on a finite domain 


governed by 


dg 8g 
Ot i Ox? 


= ô(x — €)d(t—7), 0<a2,€<L, 0<t,7, (5.5.35) 
with the boundary conditions 
gx(0,t],7) =9, ge(L,t€,7) + hg(L,tl,7)=0, 0<t, (5.5.36) 


and the initial condition g(x, 0|€,7) = 0, 0 < x < L, yields the Sturm-Liouville problem 
that we must now solve: 


g"() + r(x) =0, 90) =0, g (L)+ holL) = 0. (5.5.37) 
The nth orthonormal eigenfunction for Equation 5.5.37 is 


2(k2 + h?) 


Prlz) = TRE 12) +h cos(knx), (5.5.38) 
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where k,, is the nth root of ktan(kL) = h. We also used the identity that (k2 +h?) sin?(k,h) 
=h?. Substituting Equation 5.5.38 into Equation 5.5.30, we finally obtain 


e Example 5.5.4 


Let us use Green’s functions to solve the heat equation 


Ou u 
SS =a o 0 L, O<t 5.5.40 
Eo Ba? << L, <t, ( ) 


subject to the boundary conditions 
u(0,t)=0, u(L,t)=t, 0<t, (5.5.41) 


and the initial condition 
u(az,0) = 0, 0<a<L. (5.5.42) 


Because there is no source term, Equation 5.5.6 simplifies to 


u(x,t) = è | (g(a, t|L, rT) ue(L, T) — u(L, 7) ge (a, tL, 7)] dr (5.5.43) 
L 


— è f [g(x, t|0, T)ue(0, T) — u(0, T)ge(x, t|0, T)] dr +f (€, 0)g(a, t|€, 0) dé. 
0 0 


The Green’s function that should be used here is the one given by Equation 5.5.34. Further 
simplification occurs by noting that g(a,t|0,7) = g(a,t|L,r) = 0 as well as u(0,7) = 
u(€,0) = 0. Therefore we are left with the single integral 


u(a,t) = -a | u(L,T)gelx, t|L, T) dr. (5.5.44) 


Upon substituting for g(a, t| L, T) and reversing the order of integration and summation, 


Ee L t 2,22 
u(x,t) = — = (-1)"nsin(*) f rexp|* = (=) dt (5.5.45) 

n=1 

2L? Z J a? n?n? an? nr i 

273 ri sin( T Ja| T3 (T ) ( T2 1) i (5.5.46) 
=1 

Pisa (-1)” . (naa, [a?n?x*t an? nt 

“273 “3 sin( T, ) T2 1+ exp| — T2 ; (5.5.47) 


Figure 5.5.2 illustrates Equation 5.5.47. This solution could also have been found using 
Duhamel’s integral. 
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au (x, pi? 


Figure 5.5.2: The temperature distribution within a bar when the temperature is initially at zero and 
then the ends are held at zero at x = 0 and t at x = L. 


e Example 5.5.5: Heat equation within a cylinder 


In this example, we find the Green’s function for the heat equation in cylindrical 
coordinates 


0g a? Of Ag _ O(r — p)d(t —T) 
Ot r Or "ar 7 Qrr 


; O0<rp<b, 0<t,7, (5.5.48) 


subject to the boundary conditions lim,_,9 |g(r, |p, T)| < 00, g(b, t|p, T) = 0, and the initial 
condition g(r, 0|p,7) = 0. 
As usual, we begin by taking the Laplace transform of Equation 5.5.48, or 


1d / dG 8 oF 
= : 0.4 
rdr (« dr ) a? G ee p) Can 
Next we re-express ô(r — p)/r as the Fourier-Bessel expansion 


d(r =p) _ y Avda), (5.5.50) 


where kn is the nth root of Jo(k) = 0, and 


A= ITE a ar P) i (knr/b) rd r= ee (5.5.51) 


so that 


1d / dG s Jolknp/b)Jo(k nt /b) 
rdr (« dr ) oo 7 T a J? (k a bee?) 
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bÊ g(r,tlp,t) 


0.15 
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a (t-t)/b* 
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Figure 5.5.3: The Green’s function, Equation 5.5.54, for the axisymmetric heat equation, Equation 5.5.48, 
with a Dirichlet boundary condition at r = b. Here p/b = 0.3 and the graph starts at a? (t — r)/b? = 0.001 
to avoid the delta function at t — T = 0. 


The solution to Equation 5.5.52 is 


er (kn p/b) Jo(k nT /b) 
G(r, s|p,7) 72 CETINA (5.5.53) 


Taking the inverse of Equation 5.5.53 and applying the second shifting theorem, 


g(r, tlp, T) = a 3 >» ake erb, ma KEET, (5.5.54) 


n=1 


See Figure 5.5.3. 
If we modify the boundary condition at r = b so that it now reads 


gr(b, tlo, 7) + hg(b, tlp, 7) = 0, (5.5.55) 
where h > 0, our analysis now leads to 


H(t- 7)  Jo(knp/b)Jo(knr/b) «2K? (t—r)/02 
=>» en aka (t—7)/0 


t = 
g(r, |P, T) T de Ba) + J? (kn) 


(5.5.56) 


n=1 


where kp are the positive roots of k Jı (k) — hb Jo(k) = 0. If h = 0, we must add 1/(7b7) to 
Equation 5.5.56. 


Problems 


1. Find the free-space Green’s function for the linearized Ginzburg-Landau equation”? 
07g 
Ou? 


+v ag — b = (x — £)d(t — T), —co < z, < œ, 0 < t,T, 


20 See Deissler, R. J., 1985: Noise-sustained structure, intermittency, and the Ginzburg-Landau equation. 
J. Stat. Phys., 40, 371-395. 
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with b > 0. 
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Step 1: Taking the Laplace transform of the partial differential equation, show that it 


reduces to the ordinary differential equation 


G dG ests 
bg v teG sG = —ô(x — E)e*". 


Step 2: Using Fourier transforms, show that 


a e787 oo etk(x—€) i 
ees 27T / s +ikv+bk? -a ” 


—00O 


or 
ealt-T) 


g(x, tl, T) = 


Step 3: Evaluate the second integral and show that 


ee- H(t — 7T) [x — £ — v(t — 7)? 
RAE exp } i 


g(x, tl, T) = 


2. Use Green’s functions to show that the solution?! to 


ðu Ou 


eyed a9? 0< i 
at” Aa? 7 


subject to the boundary conditions 
u(0,t)=0, lim u(a,t) > 0, 0<t, 
«LOO 


and the initial condition 


u(x,0) = f(x), 0< z<, 
i ?/(4a°t) 
e 3 a XLT 2 2 
t) = 4 inh({ — —7*/(4a*t) qr. 
u(x,t) Ali | f(r) sin (sae T 


HEA a et) cost kx — € — v(t — 7)]} dk. 


3. Use Equation 5.5.30 to construct the Green’s function for the one-dimensional heat 
equation 9: — Jz = O(a — €)d(t — T) for 0 < x < L, 0 < t, with the initial condition that 
g(x, O|§,7) =0, 0 < a < L, and the boundary conditions that g(0,t|€,7) = gx(L, tl/€,7) = 0 


for 0 < t. Assume that L £7. 


4. Use Equation 5.5.30 to construct the Green’s function for the one-dimensional heat 
equation gt — Grr = O(a — €)d(t — T) for 0 < x < L, 0 < t, with the initial condition that 


21 See Gilev, S. D., and T. Yu. Mikhailova, 1996: Current wave in shock compression of matter in a 


magnetic field. Tech. Phys., 41, 407—411. 
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g(x, O|§,7) = 0, 0 < x < L, and the boundary conditions that g,(0, t|€,7) = gx(L, t/€,7) = 0 
for 0 < t. 


5. Use Equation 5.5.43 and the Green’s function given by Equation 5.5.34 to find the 
solution to the heat equation uz = ug, for 0 < x < L, 0 < t, with the initial data 
u(z,0) = 1, 0 < a < L, and the boundary conditions u(0,t) = e~' and u(L,t) = 0 when 
0<t. 


6. Use Equation 5.5.43 and the Green’s function that you found in Problem 3 to find the 
solution to the heat equation uz = Ugg for O < x < L, 0 < t, with the initial data u(x, 0) = 1, 
0 < x < L, and the boundary conditions u(0,t) = sin(t) and uz(L,t) = 0 when 0 < t. 


7. Use Equation 5.5.43 and the Green’s function that you found in Problem 4 to find the 
solution to the heat equation ut = Ugg for O < x < L, 0 < t, with the initial data u(x, 0) = 1, 
0 < x < L, and the boundary conditions us(0,t) = 1 and uz(L,t) = 0 when 0 < t. 


8. Find the Green’s function for 
82 
— — gq? +a? k?’g = ô(x — Est- r), O0<z,E<L, 0<t,T, 
xv 
subject to the boundary conditions 


and the initial condition 
g(a, O|€,7) = 0, 0<2¢<L, 


where a and k are real constants. 
5.6 HELMHOLTZ’S EQUATION 


In the previous sections, we sought solutions to the heat and wave equations via Green’s 
functions. In this section, we turn to the reduced wave equation 


u u 
J2 + By? H Au = — f(x,y). (5.6.1) 


Equation 5.6.1, generally known as Helmholtz’s equation, includes the special case of Pois- 
son’s equation when À = 0. Poisson’s equation has a special place in the theory of Green’s 
functions because George Green (1793-1841) invented his technique for its solution. 

The reduced wave equation arises during the solution of the harmonically forced wave 
equation?” by separation of variables. In one spatial dimension, the problem is 


Ou 1 8u 
Ox? 2 ot 


= -f (2)e (5.6.2) 


Equation 5.6.2 occurs, for example, in the mathematical analysis of a stretched string over 
some interval subject to an external, harmonic forcing. Assuming that u(x,t) is bounded 


22 See, for example, Graff, K. F., 1991: Wave Motion in Elastic Solids. Dover Publications, Inc., Section 
1.4. 
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everywhere, we seek solutions of the form u(x,t) = y(x)e~**’. Upon substituting this 
solution into Equation 5.6.2 we obtain the ordinary differential equation 


y" +key = —f(2), (5.6.3) 


where kê = w*/c?. This is an example of the one-dimensional Helmholtz equation. 
Let us now use Green’s functions to solve the Helmholtz equation, Equation 5.6.1, 
where the Green’s function is given by the Helmholtz equation 


2 
Ta ES +g = -ile -e)a n): (5.6.4) 


The most commonly encountered boundary conditions are 
e the Dirichlet boundary condition, where g vanishes on the boundary, 


e the Neumann boundary condition, where the normal gradient of g vanishes on the bound- 
ary, and 


e the Robin boundary condition, which is the linear combination of the Dirichlet and Neu- 
mann conditions. 


We begin by multiplying Equation 5.6.1 by g(x, y|€,7) and Equation 5.6.4 by u(x, y), sub- 
tract and integrate over the region a < x < b,c < y < d. We find that 


u(£, n) = f i | “{o(e,ulésm E v) a J 


g(x O07 g(x 
-uy | uig LEMET) aedy 


d b 
+ | i F(a, yl, le, n) de dy (5.6.5) 


-SÈ lonen” | - Z feep PE) aza 
+f LP fote. ie MED] - fuent], ava 


ff f(x, y)g(z, ylE, n) dz dy eee 
= f [iesen UEN) ua, y BE HED) w 
+f fo (x, ¥l€.7) we i -ue E E] i 
i [ T f(x, y)g(x, yl€, n) dz dy. (5.6.7) 


Because (€,7) is an arbitrary point inside the rectangle, we denote it in general by (x,y). 
Furthermore, the variable (x,y) is now merely a dummy integration variable that we now 
denote by (€,7). Upon making these substitutions and using the symmetry condition 
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g(x, ylE,n) = g(€, nlx, y), we have that 


: UW x£ ¿=b 
uzy) = f lates ules) ee — ule, EE | d 


o£ ag i 
b ‘ 2 
+f oyle nEn = ule, Sle dé 
a an On PEN 
d b 
+f | F(E, n)g(2, ylE, n) ag dn. (5.6.8) 


Equation 5.6.8 shows that the solution of Helmholtz’s equation depends upon the sources 
inside the rectangle and values of u(x, y) and (Ou/Ox, 0u/Oy) along the boundary. On the 
other hand, we must still find the particular Green’s function for a given problem; this 
Green’s function depends directly upon the boundary conditions. At this point, we work 
out several special cases. 


1. Nonhomogeneous Helmholtz equation 
and homogeneous Dirichlet boundary conditions 


In this case, let us assume that we can find a Green’s function that also satisfies the 
same Dirichlet boundary conditions as u(x, y). Once the Green’s function is found, then 
Equation 5.6.8 reduces to 


d b 
u(t, y) = J f FE, Mgl, vlé, n) dédn. (5.6.9) 


A possible source of difficulty would be the nonexistence of the Green’s function. From 
our experience in Section 5.2, we know that this will occur if A equals one of the eigenvalues 
of the corresponding homogeneous problem. An example of this occurs in acoustics when 
the Green’s function for the Helmholtz equation does not exist at resonance. 


2. Homogeneous Helmholtz equation 
and nonhomogeneous Dirichlet boundary conditions 


In this particular case, f(x,y) = 0. For convenience, let us use the Green’s function 
from the previous example so that g(x, y|€,7) = 0 along all of the boundaries. Under these 
conditions, Equation 5.6.8 becomes 


b 
u(e.y) == | uep AE i= fru (En) g(x, ylen) | a (5.6.10) 


OE lees 


Consequently, the solution is determined once we compute the normal gradient of the 
Green’s function along the boundary. 


3. Nonhomogeneous Helmholtz equation 
and homogeneous Neumann boundary conditions 


If we require that u(x, y) satisfies the nonhomogeneous Helmholtz equation with homo- 
geneous Neumann boundary conditions, then the governing equations are Equation 5.6.1 


Green’s Functions 269 


and the boundary conditions us = 0 along x = a and x = b, and uy = 0 along y = c and 
y =d. Integrating Equation 5.6.1, we have that 


[| oO] ays f ES aeo] m 


sa f [utenaray=— f f Hendra (5.6.11) 


Because the first two integrals in Equation 5.6.11 must vanish in the case of homogeneous 
Neumann boundary conditions, this equation cannot be satisfied if A = 0 unless 


ie f(x,y) dx dy = 0. (5.6.12) 


A physical interpretation of Equation 5.6.12 is as follows: Consider the physical process 
of steady-state heat conduction within a rectangular region. The temperature u(2,y) is 
given by Poisson’s equation 

u Ou 
da? * Oy? 


where f(x,y) is proportional to the density of the heat sources and sinks. The boundary 
conditions uz(a,y) = uz(b, y) = 0 and u,(x,c) = u,(xz,d) = 0 imply that there is no heat 
exchange across the boundary. Consequently, no steady-state temperature distribution can 
exist unless the heat sources are balanced by heat sinks. This balance of heat sources and 
sinks is given by Equation 5.6.12. 

Having provided an overview of how Green’s functions can be used to solve Poisson 
and Helmholtz equations, let us now determine several of them for commonly encountered 
domains. 


= — f(x,y), (5.6.13) 


e Example 5.6.1: Free-space Green’s function for the one-dimensional Helmholtz equation 
Let us find the Green’s function for the one-dimensional Helmholtz equation 
g + kg = —ô(x — £), —00 < T, < 00. (5.6.14) 
If we solve Equation 5.6.14 by piecing together homogeneous solutions, then 
gle) = Ae eC 4 Beles), (5.6.15) 


for x < €, while 


g(alé) = Ce~*ole-8) 4 Dettol-8), (5.6.16) 


for € < x. 

Let us examine Equation 5.6.15 more closely. The solution represents two propagating 
waves. Because x < &, the first term is a wave propagating out to infinity, while the second 
term gives a wave propagating in from infinity. This is seen most clearly by including the 
et term into Equation 5.6.15, or 


g(aléje7*" = Ae ikoe) ivt + Betko(a—§)—iwt (5.6.17) 


Because we have a source only at x = €, solutions that represent waves originating at infinity 
are nonphysical and we must discard them. This requirement that there are only outwardly 
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(b) Contour forx < § 


(a) Contour forx > § 


Figure 5.6.1: Contour used to evaluate Equation 5.6.21. 


propagating wave solutions is commonly called Sommerfeld’s radiation condition.?> Similar 
considerations hold for Equation 5.6.16 and we must take C = 0. 
To evaluate A and D, we use the continuity conditions on the Green’s function: 


g(ET|E) = gE lE), and g(E*1€) — g' (E-|6) = -1, (5.6.18) 
or 
A=D, and ikoD+ikoA = -1. (5.6.19) 
Therefore, 
g(al€) = A (5.6.20) 


We can also solve Equation 5.6.14 by Fourier transforms. Assuming that the Fourier 

transform of g(x|€) exists and denoting it by G(k|&), we find that 
Gi ew tké i 1 co etk(w—€) dk 
= er S o k: 6.21 
WO=pae md D=) Gop (5.6.21) 

Immediately we see that there is a problem with the singularities lying on the path of 
integration at k = +ko. How do we avoid them? 

There are four possible ways that we might circumvent the singularities. One of them 
is shown in Figure 5.6.1. Applying Jordan’s lemma to close the line integral along the real 
axis (as shown in Figure 5.6.1), 


1 e'2(@—§) 
g(x|§) = z$ 2 ke dz. (5.6.22) 


23 Sommerfeld, A., 1912: Die Greensche Funktion der Schwingungsgleichung. Jahresber. Deutschen 
Math.- Vereinung, 21, 309-353. 
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Free-Space Green’s Function for the Poisson and Helmholtz Equations 


Dimension Poisson Equation Helmholtz Equation 
One no solution g(a) = —— etkola—s| 
Do 
In(r) i (1) 
Two glz ylen) = -s glz, ylé n) = 4 Ho (kor) 


r= Væ- +u- 


Note: For the Helmholtz equation, we have taken the temporal forcing to be e~*”* and ko = w/c. 


For a < £, 
iz(x—£) ; 
f 5 e . = 1 —iko(x— 
g(a|g) = —i Res] S zE ro a ako o( 8 (5.6.23) 
while TART 
a ERA U ikoa 
wale) = i Res] ita] = gp, (5.6.24) 


for x > E. A quick check shows that these solutions agree with Equation 5.6.20. If we try 
the three other possible paths around the singularities, we obtain incorrect solutions. 


e Example 5.6.2: Free-space Green's function for the two-dimensional Helmholtz equation 


At this point, we have found two forms of the free-space Green’s function for the one- 
dimensional Helmholtz equation. The first form is the analytic solution, Equation 5.6.20, 
while the second is the integral representation, Equation 5.6.21, where the line integration 
along the real axis is shown in Figure 5.6.1. 

In the case of two dimensions, the Green’s function”* for the Helmholtz equation sym- 
metric about the point (£, n) is the solution of the equation 


—~ 4 (44 kig = (5.6.25) 
r 


where r = „/(x — £)? + (y — n)?. The homogeneous form of Equation 5.6.25 is Bessel’s 
differential equation of order zero. Consequently, the general solution in terms of Hankel 
functions is 

g(r|ro) = AHS” (kor) + BH (kor). (5.6.26) 


Why have we chosen to use Hankel functions rather than Jo(-) and Yo(-)? As we argued 
earlier, solutions to the Helmholtz equation must represent outwardly propagating waves 
(the Sommerfeld radiation condition). If we again assume that the temporal behavior 
is e~*“* and use the asymptotic expressions for Hankel functions, we see that H (kor) 
represents outwardly propagating waves and B = 0. 


24 For an alternative derivation, see Graff, K. F., 1991: Wave Motion in Elastic Solids. Dover Publica- 
tions, Inc., pp. 284-285. 
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What is the value of A? Integrating Equation 5.6.26 over a small circle around the 
point r = 0 and taking the limit as the radius of the circle vanishes, A = i/4 and 


a 
g(r\ro) = Ho” (hor). (5.6.27) 


If a real function is needed, then the free-space Green’s function equals the Neumann 
function Yo(kor) divided by —4. 


e Example 5.6.3: Free-space Green's function for the two-dimensional Laplace equation 


In this subsection, we find the free-space Green’s function for Poisson’s equation in two 
dimensions. This Green’s function is governed by 


10 / 0g 18g — d(r — p)d(0 — 0") 
r Or (« 2) r2 002 r f e028) 


If we now choose our coordinate system so that the origin is located at the point source, 
r = „y(x — £)? + (y — n)? and p = 0. Multiplying both sides of this simplified Equation 
5.6.28 by rdrd@ and integrating over a circle of radius €, we obtain —1 on the right side 
from the surface integration over the delta functions. On the left side, 


do = —1. (5.6.29) 


The Green’s function g(r, 00, 6’) = —In(r)/(27) satisfies Equation 5.6.29. 
To find an alternative form of the free-space Green’s function when the point of exci- 
tation and the origin of the coordinate system do not coincide, we first note that 


1 oo ; ; 
_ al) — in(@—6') 
6(0 — 0") = a ae : (5.6.30) 
This suggests that the Green’s function should be of the form 


Co 


g(r, \p,8') = X glr). (5.6.31) 


n=— oo 


Substituting Equation 5.6.30 and Equation 5.6.31 into Equation 5.6.29, we obtain the or- 
dinary differential equation 


1 d / dgn n? d(r — p) 
— . 5.6.32 
rdr ¢ dr ) r2 y Qrr ( ) 


The homogeneous solution to Equation 5.6.32 is 


a, O<r<p, 


go(r|p) = {5 Ine); eens (5.6.33) 
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and (r/p) 
_ Jelr/p)”, O<r<p, 
WOE ee (5.6.34) 
ifn #0. 
At r = p, the g,,’s must be continuous, in which case, 
a = bln(p), and c=d. (5.6.35) 
On the other hand, 
+ 
deal. © 1 
— = —— 5.6.36 
dr | 2T’ ( ) 
r=p 
Á In(p) 1 1 
= — TP) = —— = = — 
a oe b 50 and c=d Tar (5.6.37) 
Therefore, 
(r, |p, 0") = ls eae S ALE ae — 6’) (5.6.38) 
NE 27 mén r> , it 


where rs, = max(r, p) and re = min(r, p). 
We can simplify Equation 5.6.38 by noting that 


pe 0 — 0')] 
In[1 + p? — 2pcos(0 - 0')] = -2Y £? eum 5.6.39 
nji +p pcos( )] a = , ( ) 
if |p| < 1. Applying this relationship to Equation 5.6.38, we find that 


g(r, |p, 0") = 


1 
re In[r? + p? — 2rpcos(0 — 6’)] . (5.6.40) 


Note that when p = 0 we recover g(r, 6|0, 6’) = —In(r)/(27). 


e Example 5.6.4: Two-dimensional Poisson equation over a rectangular domain 


Consider the two-dimensional Poisson equation 


u Ou 
3x2 F Oy? = —f (x,y). (5.6.41) 


This equation arises in equilibrium problems, such as the static deflection of a rectangular 
membrane. In that case, f(x,y) represents the external load per unit area, divided by the 
tension in the membrane. The solution u(x, y) must satisfy certain boundary conditions. 
For the present, let us choose u(0, y) = u(a, y) = 0, and u(a#,0) = u(x, b) = 0. 

To find the Green’s function for Equation 5.6.41 we must solve the partial differential 
equation 


| =-d(a—E)d(y—), O<aE<a, 0<y,n7<b, (5.6.42) 


subject to the boundary conditions 


9(0, yl€,7) = g(a, ylé, n) = g(x, O|€, n) = g(a, bE, n) = 0. (5.6.43) 
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a b 
TOE f | oa, vlé, n) f(E, n) dn dë. (5.6.44) 


One approach to finding the Green’s function is to expand it in terms of the eigenfunc- 
tions y(x, y) of the differential equation 


2 
+i =—hy, (5.6.45) 


and the boundary conditions, Equation 5.6.43. The eigenvalues are 


nen mêr? 


Anm = os + Oa (5.6.46) 
where n = 1,2,3,..., m=1,2,3,..., and the corresponding eigenfunctions are 
Pnm(@,y) = sin(“**) sin( 2#) ; (5.6.47) 
a 


Therefore, we seek g(x, y|€,7) in the form 


(x, yl, n) a. S Anm sin( ZZ Z) sin( 2#) i (5.6.48) 


n=1m=1 


Because the delta functions can be written 


d(a — €)d(y—n) = “ ‘5 So sin(“*) sin( 2) sin(“*) sin (2) , (5.6.49) 


n?n? mêr? 4 . (nmE\ . (mm 
(= + T) Anm = ob sin( 7 ) sin( 5 ) ; (5.6.50) 


after substituting Equations 5.6.48 and 5.6.49 into Equation 5.6.42, and setting the corre- 
sponding harmonics equal to each other. Therefore, the bilinear formula for the Green’s 
function of Poisson’s equation is 


sin (272) sin (27E) sin (2) sin (270) 


4 co loc) 
g(x, ylé.m) Sa a Soy Oey l (5.6.51) 


Thus, solutions to Poisson’s equation can now be written as 


uls, y) => D n?n? Ja? K 2792 sin( 2) sin( Z8), (5.6.52) 


n=1 m= 
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where anm are the Fourier coefficients for the function f(x,y): 


nm = =f i f(x,y) sin(*) sin n( 8) dy dx. (5.6.53) 


Another form of the Green’s function can be obtained by considering each direction 
separately. To satisfy the boundary conditions along the edges y = 0 and y = b, we write 
the Green’s function as the Fourier series 


(x, yl€,7) 9 Gm(a|€) sin( 2) ; (5.6.54) 
mal 


where the coefficients Gm(x|£) are left as undetermined functions of x, €, and m. Substi- 
tuting this series into the partial differential equation for g, multiplying by 2sin(nay/b)/b, 
and integrating over y, we find that 


CG, nn 2 nt Sa 


Sor ~ ay Gn = —F sin ê). (5.6.55) 


This differential equation shows that the expansion coefficients Gn (x|£) are one-dimensional 
Green’s functions; we can find them, as we did in Section 5.2, by piecing together homo- 
geneous solutions to Equation 5.6.55 that are valid over various intervals. For the region 
0 < x < £, the solution to this equation that vanishes at x = 0 is 


NTL 


Gn (zl) = An sinh(““*) (5.6.56) 


where A,, is presently arbitrary. The corresponding solution for € < x < ais 


G,,(2|g) = Bn sinh esa 2)) (5.6.57) 
Note that this solution vanishes at x = a. Because the Green’s function must be continuous 
at x =, 
An sin ( 27E) = By sin [2762] . (5.6.58) 
On the other hand, the appropriate jump discontinuity of G% (x|) yields 
nT nr(a— &) nT nmmE\ 2, (an 
= Bn cosh| 5 5 An cosh( 5 ) =—;sin(——)- (5.6.59) 


Solving for A, and Bn, 


= 2. (nTn) sinh[nr(a — €)/0] 
2 NT sin( b ) sinh(nra/b) ’ (5.6.60) 
i 2 inh(nm€/b) 
(nan) sinh(nr 
oe nT sin( b ) sinh(nra/b)` (5.6.61) 


This yields the Green’s function 


œ sinh|nr(a — x> )/b]sinh(nrz</b) . (nany . nay 
2 n sinh(nra/b) sin( b ) sia ( 28), (5.6.62) 


g(x, yl, n) 


T aci 
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0.5 


Ox ym) 


Figure 5.6.2: The Green’s function, Equation 5.6.62 or Equation 5.6.63, for the planar Poisson equation 
over a rectangular area with Dirichlet boundary conditions on all sides when a = b and €/b = n/b = 0.3. 


where xy = max(x,é) and x< = min(z,€). Figure 5.6.2 illustrates Equation 5.6.62 in the 
case of a square domain with €/b = n/b = 0.3. 
If we began with a Fourier expansion in the y-direction, we would have obtained 


Haden > sinh|mr(b — y> )/a] sinh(mry</a) sin( "=" sin( 277) _ (5.6.63) 


m sinh(mmb/a) 


or 


where ys = max(y,7) and ye = min(y, n). 
e Example 5.6.5: Two-dimensional Helmholtz equation over a rectangular domain 


The problem to be solved is 


o? o? 
= + n t kog = —ô(z — EJô(y — 1), (5.6.64) 


where 0 < x, < a, and 0 < y,7 < b, subject to the boundary conditions that 


g(0, ylé, n) = g(a, ylé, n) g g(x, O|€, n) = g(x, IE, n) =0. (5.6.65) 


We use the same technique to solve Equation 5.6.64 as we did in the previous example 
by assuming that the Green’s function has the form 


glz, ylė n) = 5 Gm (2|€) sin( 2#) ; (5.6.66) 
m=1 


where the coefficients Gm (x|E) are undetermined functions of x, €, and 7. Substituting this 
series into Equation 5.6.64, multiplying by 2sin(nay/b)/b, and integrating over y, we find 
that 


Gy, nêr? 2 2 . (nin 
Tr ( 52 rè) Gnr = = sin( 5 OG £): (5.6.67) 
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g(x ym) 


Figure 5.6.3: The Green’s function, Equation 5.6.72, for Helmholtz’s equation over a rectangular region 
with a Dirichlet boundary condition on the sides when a = b, koa = 10, and €/a = n/a = 0.35. 


The first method for solving Equation 5.6.67 involves writing 


5(x —£) = 7 > sin (==) sin( 22) (5.6.68) 


a 


and 


G= : ` anm sin( 272). (5.6.69) 


Upon substituting Equations 5.6.68 and 5.6.69 into Equation 5.6.67, we obtain 
œ 2,2 2,2 
2 M T N T r MTE 
S (8M ans (22) 
=: SS . ("=") . (mnE\ . (™*) (5.6.70) 
= ab sin b sin F sin . .O. 


a 


Matching similar harmonics, 


4sin(m7€/a) sin(nan/b) 
ab(m?n? /a? + n?r?/b? — ke)’ 


and the bilinear form of the Green’s function is 


Anm e 


(5.6.71) 


4 S D sin(mm€/a) sin(nan/b) sin(mrz/a) sinning N (5.6.72) 


glz, yig, n) = ab mn? /a2 + n272/b? — k? 


See Figure 5.6.3. The bilinear form of the Green’s function for the two-dimensional Helm- 
holtz equation with Neumann boundary conditions is left as Problem 8. 
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As in the previous example, we can construct an alternative to the bilinear form of the 
Green’s function, Equation 5.6.72, by writing Equation 5.6.67 as 

Gn 

dx? 


2 = 2, (NTN 
- k2Gn = = sin( k UG £), (5.6.73) 


where k2 = n?n? /b? — k2. The homogeneous solution to Equation 5.6.73 is now 


Pi A, sinh(k,2), O0<a<&, 
Gn(al€) = E smile. eee a, (eT) 
This solution satisfies the boundary conditions at both endpoints. 
Because G,,(x|€) must be continuous at x = £, 
An sinh(kn€) = Bn sinh[k,(a — £)]. (5.6.75) 
On the other hand, the jump discontinuity involving G’, (z|€) yields 
2 
-En Bn cosh[kn(a — €)] — kn An cosh(kn£) = —F sin( 222) . (5.6.76) 
Solving for An and Bn, 
2 . ¢nmny sinh|kn(a — £)] 
Apis 5.6.77 
bkn sin( b ) sinh(kna) ( ) 
and nkné) 
2 nin \ sinh(k, 
By, = i : .6.78 
bkn sin( b ) sinh(kna) (5-6-78) 
This yields the Green’s function 
N 
_ 2 sin[kn(a — £> )]sin(xnz<) . (nm) . (NTY 
g(x, yl, n) = b 2 E A sin( 5 ) sin(™) 
2 & sinh|kn(a — z>)]sinh(knz<) . (nan). (nay 
} 6.7 
b ey, kn sinh(k,a) sin( b ) sin( b ) i Pe?) 


where zy = max(z, €) and x< = min(x,€). Here N denotes the largest value of n such that 
k2 < 0, and K2 = kê — n?n? /b?. If we began with a Fourier expansion in the y direction, 
we would have obtained 


a sin| Kn (b — in(Km MT MT 
g(x, yé, n) D [Km (b= y>)]sin(emy<) sin( = sin( 22) 


“a Km Si0(Kmd) 


2 sinh[km(b — y>)] et sin ( 2E) sin( =), (5.6.80) 


a asi km sinh(kmb) 


m=1 


where M denotes the largest value of m such that k2, < 0, k?, = m?n? /a? — kê, 62, = 


kg — m?n?/a?, y< = min(y, n), and ys = max(y, n). 
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e Example 5.6.6: Two-dimensional Helmholtz equation over a circular disk 


In this example, we find the Green’s function for the Helmholtz equation when the 
domain consists of the circular region 0 < r < a. The Green’s function is governed by the 
partial differential equation 


10 / 0g 18g 15 d(r—p)d(0— 6’) 
are (2) + 72 902 + kĝg = - ; (5.6.81) 


where 0 < r,p < a, and 0 < 6,6’ < 2r, with the boundary conditions 
lim |g(r,6|p,6")| <œ,  gla,0lp,0)=0, 0 < 0,0" < 2r. (5.6.82) 
r= 


The Green’s function must be periodic in 0. 
We begin by noting that 


PN: es F 1 T 1 
OG oad 0 ) = on + = ee TEA 0 )] = a a cos[n(@ Soa 0 JE (5.6.83) 
Therefore, the solution has the form 
a(r,0\p,0') = Y galrlp)cos[n(0 = 6’). (5.6.84) 


Substituting Equation 5.6.83 and Equation 5.6.84 into Equation 5.6.81 and simplifying, we 


find that TEE F 5 ) 
I\_ n amt 
(« : ) “a gn + kon = -> ——. (5.6.85) 


rdr 2nr 


The solution to Equation 5.6.85 is the Fourier-Bessel series 


= Yo Anma ( seit (5.6.86) 


where knm is the mth root of J,(k) = 0. Upon substituting Equation 5.6.86 into Equation 
5.6.85 and solving for Anm, we have that 


(ke kê fo Ann = a Ea erent h o(r — pn ( z) dr, (5.6.87) 
or k 
T (Kim — koa?) J'n (Km) 


Thus, the Green’s function?? is 


g(r, lp, 0") = Do Sh (k2 =o cos[n(8 — 6’)]. (5.6.89) 


n=—oom=1 


25 For an example of its use, see Zhang, D. R., and C. F. Foo, 1999: Fields analysis in a solid magnetic 
toroidal core with circular cross section based on Green’s function. IEEE Trans. Magnetics, 35, 3760-3762. 
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g(r,8/p,8’) 


Wa xa 


Figure 5.6.4: The Green’s function, Equation 5.6.89, for Helmholtz’s equation within a circular disk with 
a Dirichlet boundary condition on the rim when koa = 10, p/a = 0.35V2, and 6’ = T/4. 


See Figure 5.6.4. 
Problems 


1. Using a Fourier sine expansion in the x-direction, construct the Green’s function governed 
by the planar Poisson equation 
ə? 3? 
fa ia 
Ox? 8y? 


ô(x — éjôly =n), O<a,E<a, —00 < y,n < o, 
with the Dirichlet boundary conditions 

g(0,ylé, n) = gla y|, n) =0, —co<y<oo, 
and the conditions at infinity that 


lim g(x, yl, n) > 0, 0<z<a. 


|yl—> 20 


2. Using a generalized Fourier expansion in the x-direction, construct the Green’s function 
governed by the planar Poisson equation 
ag | Og 


pe tg Oe es. TEA Sere ta See 


with the Neumann and Dirichlet boundary conditions 
gx (0, yl€, n) = g(a, yl€,7) =0, -oo<y<o, 
and the conditions at infinity that 


lim g(a, yl, n) > 0, 0<a4<a. 


lyl+oo 
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3. Using a Fourier sine expansion in the y-direction, show that the Green’s function governed 
by the planar Poisson equation 
3g g 
Ox? ` Əy 


s(x — Ely- n), VOR BES a, 0<y,n <b, 


with the boundary conditions 


g(x, O|€,7) = g(x, b|, n) =0, 


and 
g(0, yl, n) = g(a, yl, n) + 8 gla yl n) =0, B20, 
is 
2 sinh(va<) {v cosh|v(a — z>)| + Bsinh[v(a—2s)]} . pnany . (nry 
(2, yls,n) a v? cosh(va) + 8v sinh(va) sin( b ) sin( b ) i 


where v = nz/b, x> = max(zx, £), and z< = min(z, £). 


4. Using the Fourier series representation of the delta function in a circular domain: 
5(0 — 6’) )=>+2 = Lewin (0 — 6’) 0< 0,6’ < 2r, 


construct the Green’s function governed by the planar Poisson equation 


1a ( z) 18g — 5(r — p)d(0 — 6’) 


r Or "Or r2 Ob r i 


where a < r,p < b, and 0 < 6,0’ < 27, subject to the boundary conditions g(a, 6|p, 6’) = 
g(b, 0|p, 0’) = 0 and periodicity in 0. 


5. Construct the Green’s function governed by the planar Poisson equation 


10/ 0g 18g — d(r — p)d(0 — 0") 
r Or \" Or 


r2 002 r 


where 0 < r, p < co, and 0 < 6,0’ < 8, subject to the boundary conditions that g(r, 0ļp, 6’) 
= g(r, 8lp,0') = 0 for all r. Hint: 


N] 


6. Construct the Green’s function governed by the planar Poisson equation 


10/ 0g 18g  d(r—p)d(0- 0’) 
r Or \" Or 


r2 002 r , 
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where 0 < r,p < a, and 0 < 0,6’ < B, subject to the boundary conditions g(r, 0|p, 0’) = 
g(r, B\p, 0’) = g(a, O|p, 6’) = 0. Hint: 


2— nr? \ . (nnd 
UPSR De a j) 


7. Using a Fourier sine series in the z-direction and the fact that 


_ 2b Ș Jo(unb/a) Jo(unx/a) 


aS: ’ 0< b< $ 
a JC) tees 


where ju, is the kth positive root of Jo(u) = 0, find the Green’s function governed by the 
axisymmetric Poisson equation 


Og 10g 8g d(r — p)d(z-¢) 
ðr? ` r Or O22 Qrr ‘ 


where 0 < r,p < a, and 0 < z,¢ < L, subject to the boundary conditions 
g(r, 0|p,¢) = g(r, Llp,¢) =0, O<r<a, 


and 
lim |g(r,2|p,6)| < 00, g(a,2|p,6)=0, O<2<L. 


8. Following Example 5.6.5 except for using Fourier cosine series, construct the Green’s 
function?® governed by the planar Helmholtz equation 


3g Og 


jaz T yp -kog = —d(a— €)5(y—), O<a,E<a, 0<y,n< b, 


subject to the Neumann boundary conditions 


gx(0, YIE, n) = ga(a,ylE,n) =0, O<y<b, 


and 
Gy (x, Ol, n) = gy(x, O|€,n) =0, O<a<a. 


9. Using Fourier sine transforms, 


g(x, yl, n) = G(k, y|, n) sin(kx) dk, 


where 


G(k, ylé,n) = i a(x, yl£, n) sin(kz) de, 


26 Kulkarni et al. (Kulkarni, S., F. G. Leppington, and E. G. Broadbent, 2001: Vibrations in several 
interconnected regions: A comparison of SEA, ray theory and numerical results. Wave Motion, 33, 79-96) 
solved this problem when the domain has two different, constant kes. 
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find the Green’s function governed by 


| = —ô(x — €)d(y — n), 
for the quarter space 0 < x,y, with the boundary conditions 


9(0, yl€,n) = g(x, O|€,) = 0, 


and 
lim g(x, y|€,n) > 0. 
xz, y—> 0 


Step 1: Taking the Fourier sine transform in the x direction, show that the partial differ- 
ential equation reduces to the ordinary differential equation 


ËG i 
dy -PG=- sin(k€)d(y — n), 


with the boundary conditions 


G(k,0|§,7) =0, and lim G(k, ylé,n) > 0. 
y CO 


Step 2: Show that the particular solution to the ordinary differential equation in Step 1 is 


G,(k, ylé, n) = E i y 


You may want to review Example 5.2.8. 


Step 3: Find the homogeneous solution to the ordinary differential equation in Step 1 so 
that the general solution satisfies the boundary conditions. Show that the general solution 


is 
_ sin(k§) f —kly-nl —k(ytn) 
G(R, yl, n) = Ge le =e ] ; 


Step 4: Taking the inverse, show that 


i k 
g(x, yl, n) = = f (tn — Hot] sin(k€) sin(ka) © 
T Jo 


Step 5: Performing the integration,?” show that 


_ 1) file 9? + y= WP lle + 
glz, yl, n) = ante QF yt nero? 


27 Gradshteyn, I. S., and I. M. Ryzhik, 1965: Table of Integrals, Series, and Products. Academic Press, 
Section 3.947, Formula 1. 
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10. Find the free-space Green’s function?® governed by 


ag Og 
ðr? ` Oy? 


g = —0(x — ¿)ô(y — n), —00 < T, Y, E, < co. 


Step 1: Introducing the Fourier transform 


Waden -5f G(k, ylé. ne** dk, 


where 


G(k, ylé,n) = J g(x, ylE, ne ** da, 


—oCo 


show that the governing partial differential equation can be transformed into the ordinary 
differential equation 
dG 
dy? 


(k? +1) G= eS S (y — n). 


Step 2: Introducing the Fourier transform in the y-direction, 


G(k, yl, n) ->f G(k, lE, ne" dé, 


where 


Tik, elé, n) = i G(k, ylé men dy, 


solve the ordinary differential equation in Step 1 and show that 


A e` thé L(y—n) x 
Gk, wg.) = S. PET 


Step 3: Complete the problem by showing that 


etk (2—8) eit(y—=n) 
g(a, yl, n) = el. f° arp ete 


2T etr" cos(0— 4) 
ie of ee —,—— k.dé dr 


© Jo(rk) Kale) ) 
d 
On Jy K2+1 CON ied 


where r = \/(x — £)? + (y—7)?, k = Kcos(0), £ = Ksin(0), x — € = rcos(y), and y—7 = 
rsin(y). You will need to use integral tables?® to obtain the final result. 


28 For its use, see Geisler, J. E., 1970: Linear theory of the response of a two layer ocean to a moving 
hurricane. Geophys. Fluid Dyn., 1, 249-272. 


29 Gradshteyn and Ryzhik, op. cit., Section 6.532, Formula 6. 
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11. Find the free-space Green’s function governed by 


3g Og ôg 
ðr? Oy? Ox 


O(a E)d(y n), = < £,Y, n < OO. 


Step 1: By introducing y(x, y|€,7) such that 


g(x, ylE,n) = P plx, ylé,n), 


show that the partial differential equation for g(x, y|€,7) becomes 


Pp Pp gp e 


Step 2: After taking the Fourier transform with respect to x of the partial differential 
equation in Step 1, show that it becomes the ordinary differential equation 


dO 
dy? 


(Er ea y= 1). 


Step 3: Introducing the same transformation as in Step 3 of the previous problem, show 
that 


ae e §/2-tk& foo (y-n) a 
( „yl n) = on ee ’ 
and 
e§/2 n 
(a, yl n) = = Eola"), 
T 


where r = \/(x — £)? + (y—1)?. 
Step 4: Using the transformation introduced in Step 1, show that 


e(e-6)/2 


g(x, ylE,n) = = 


5.7 GALERKIN METHOD 


In the previous sections we developed various analytic expressions for Green’s functions. 
We close this chapter by showing how to construct a numerical approximation. 

Finite elements can be used to solve differential equations by introducing subdomains 
known as finite elements rather than a grid of nodal points. The solution is then represented 
within each element by an interpolating polynomial. Unlike finite difference schemes that are 
constructed from Taylor expansions, the theory behind finite elements introduces concepts 
from functional analysis and variational methods to formulate the algebraic equations. 

There are several paths that lead to the same finite element formulation. The two most 
common techniques are the Galerkin and collocation methods. In this section we focus on 
the Galerkin method. This method employs a rational polynomial, called a basis function, 
that satisfies the boundary conditions. 
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We begin by considering the Sturm-Liouville problem governed by 


dn 


dx? 


+ AnVn = 0, 0<a<L, (5.7.1) 


subject to the boundary conditions Yn(0) = Yn(L) = 0. Although we could solve this 
problem exactly, we will pretend that we cannot. Rather, we will assume that we can 
express it by 


N 
z) = 5 An; fila), (5.7.2) 
j=1 
where f;(x) is our basis function. Clearly, it is desirable that f;(0) = f;(L) = 0 


How do we compute a,,;? We begin by multiplying Equation 5.7.1 by f;(x) and inte- 
grating the resulting equation from 0 and L. This yields 


L dy L 
/ fix) > dz + rn ff fi(x)Un(x) dx = 0, (5.7.3) 
0 dx 0 
where i = 1,2,3,..., N. Next, we substitute Equation 5.7.2 and find that 
S| ro fila) f; (x) dx + An fa fila) f(x jaz! Qnj = 0. (5.7.4) 
We can write Equation 5.7.4 as 
(A+ A,B)d = 0, (5.7.5) 
where 
L L 
ay= | RORE de=- | Kefe) ae, (5.7.6) 
0 0 
L 
bij = i Fi(x) fj (x) de, (5.7.7) 
0 


and the vector d contains the elements an;. 
There are several obvious choices for f; (x): 


e Example 5.7.1 


The simplest choice for fj(x) = sin(jaa/L). If we select N = 2, Equation 5.7.2 becomes 


Un(£) = Ani sin(™) + Qn2 sin( =) $ (5.7.8) 


From Equation 5.7.6 and Equation 5.7.7, 


ESO y ; i 
e=- (2) T sin( =) sin( 4) dx, i=1,2, j=1,2; (5.7.9) 


L ; ; 
w= sin( =) sin( 4) dz, i=1,2, j=1,2. (5.7.10) 
` L L 


and 
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Performing the integrations, a12 = a2, = b12 = b21 = 0, ay, = —7°/ (2L), agg = —27?°/ L, 
and bi4 = bog = L/2. 
Returning to Equation 5.7.5, it becomes 


T pe E & K (3) l (5.7.11) 


In order for Equation 5.7.11 to have a unique solution, 


=r? /2 + AnL?/2 0 
| / 0 / 2m? + Ay L?/2 | = 0. (5.7.12) 
Equation 5.7.12 yields 441 = Ag = 4n?/L?. 
In summary, 
p(x) = sin(r2/L), Sa? 7: (5.7.13) 
and 
w(x) = sin(2ra/L), dg = 4n?/L?, (5.7.14) 


with &i2 = a2; = 0. Here we have chosen that a1, = agg = 1. 


e Example 5.7.2 
Another possible choice for f;(2) involves polynomials of the form (1 — 2/L)(x/L)# 
with 7 = 1,2. Unlike the previous example, we have nonorthogonal basis functions here. 
Note that f;(0) = f;(Z) = 0. Therefore, Equation 5.7.2 becomes 
Wn (x) = Oni(1 — 2/L)(2/L) + ane(1 — 2/L)(x/L)?. (5.7.15) 


From Equation 5.7.6 and Equation 5.7.7, 


L x NE als x\I-2 aan zyj- 
u-i f HO bo-v@)*-wenG] 


2h (G0 =a aE ED 
L|i+j-1 i+j i+j i+j+lj’ 


with 7 = 1,2 and j = 1,2. Similarly, 


=f ODODE & 6718) 


1 2 1 
= L\|— — + ——— f (5.7.19) 
i+j+1 t4+94+2 t+94+8 


Performing the computations, a1, = —1/(3L), aig = a21 = —1/(6L), a22 = —2/(15L), 
bii = L/30, by = ba = L/60, and bo = L/105. 
Returning to Equation 5.7.5, it becomes 


ee er Re Gye 
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In order for Equation 5.7.20 to have a unique solution, 


| —1/3 + An L? /30  —1/6 + àn L?/60 


—1/6 + An L?/60 —2/15 + ie = (5.7.21) 


Equation 5.7.21 yields å} L? = 10 and 2L? = 42. Note how close these values of À are to 
those found in the previous example. Returning to Equation 5.7.20, we find that a11 = 1, 
Q12 = 0, Q22 = 1, and Q21 = —1/2. 

In summary, 


V(x) = (1 = =) T à = ai (5.7.22) 
ane 1 PVE x TN 2 42 
hle) = —5 (1 =) z4 (1 =) (2) Eoo (5.7.23) 


Because fj(x) are linearly independent, their use in the Galerkin expansion is quite ac- 
ceptable. However, because these functions are not particularly orthogonal, their usefulness 
will become more difficult as N increases. Consequently, the choice of orthogonal functions 
is often best. 


How do we employ the Galerkin technique to approximate Green’s functions? We begin 
by considering the inhomogeneous heat conduction problem: 


ðu Ou 
— —- 5 =Fi(a,t L t 17.24 
AE Dw (x,t), O0<a<L, 0<t, (5 ) 
with the boundary conditions 
u(0,t) = u(L, t) = 0, 0<t, (5.7.25) 


and the initial condition u(x,0) = 0,0 < x < L. 
Let us write the solution to this problem as 


N 
u(x,t) = X ava. (5.7.26) 


Direct substitution of Equation 5.7.26 into Equation 5.7.24, followed by multiplying the 
resulting equation by f;(x) and integrating from 0 to L, gives 


2 2 — d N /de z 
—An ; n n —An 3 
2 oe | filz) Jr? dz | it Aven) e p! fila yYn(x) dx 
L 
= -f filx)F (a, t) dx. (5.7.27) 
0 
Because A 
dVn + Ann = 0, (5.7.28) 


dx? 


Equation 5.7.27 simplifies to 


N 5 3 
den -Ant AE rdt = (x x ie * 
2 di J eel f fila) F(a, t) dx = F(t), (5.7.29) 
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where 1 = 1,2,...,N. 
We must now find c,. We can write Equation 5.7.29 as 


N 


den 
y T dx = F*(t), (5.7.30) 
=1 
where 
N 
Cin = 5 Qnjbji- (5.7.31) 
j=1 
Using linear algebra, we find that 
en ant n 
= Sort j; (5.7.32) 


where pn; are the elements of an array P = E`! and E = (DB)T. The arrays D and B 
consist of elements a;; and bij, respectively. Solving Equation 5.7.32, we find that 


= An + Yow f EN (nen dn. (5.7.33) 


Because u(2,0) = 0, c,(0) = 0 and A, = 0. 
We are now ready to find our Green’s function. Let us set F(x, t) = 6(a — €)d(t — T). 
Then F*(t) = f;(€)d(t — 7) and 


N 
Cn(t) = H(t — 1) > rmifi(Qe"- (5.7.34) 
i=1 
From Equation 5.7.2, Equation 5.7.26, and Equation 5.7.34, we obtain the final result that 
N N N 
g(x, tl€, 7) a H( t=T DODDA AON MACAC (5.7.35) 
n=1 j=1 i=1 


e Example 5.7.3 


In Example 5.5.2, we solved the Green’s function problem: 


ðg Og | 
a age OE) (5.7.36) 


with the boundary condition 


g(0, t|€,7) = g(L, t|, T) = 0, (5.7.37) 


and the initial condition g(x, 0|€,7) = 0. There we found the solution (Equation 5.5.34): 


g(z,t|€,7) = H(t- r dw Eyn (r) 0D, (5.7.38) 


290 Advanced Engineering Mathematics: A Second Course 


where we have the orthonormal eigenfunctions 


Wn (x) = /2/L sin(k, 2), kn = nr/L. (5.7.39) 


Let us use the basis function f;(x) = (1— x/L)(x/L) to find the approximate Green’s 
function to Equation 5.7.36. Here j = 1,2,3,..., N, 
For N = 2, we showed in Example 5.7.2 that 


= 1/30 1/60 P 1 0 
RED Ge Te as & a Ey) 
Consequently, 
L (28 14 L (28 0 
ap= = (4 p=a (i ae (SAI) 
Using Gaussian elimination, 
= 1 30 0 
= Laaa 
P=E`°=>7 E ay) l (5.7.42) 


Therefore, the two-term approximation to the Green’s function, Equation 5.7.38, is 


g(a,tlé,7) = SF (1 Dil 5) exo] | at z) 
aa hie) ee a 
n OS a ee) a 

42(t — 7) 

T 


= 


x ep] H(t—7). (5.7.43) 


For N > 2, hand computations are very cumbersome and numerical computations are 
necessary. For a specific N, we first compute the arrays A and B via Equation 5.7.17 and 
Equation 5.7.19. 


for j = 1:N 
for i= 1:N 
AG,j) = j*(j-1)/(i+j-1) - j*(j-1)/(i+j) 
+ j*(j+1)/(i+j+1) - j*(j+1)/(i+j) ; 
B(i,j) = 1/(i+j+1) - 2/(i+j+2) + 1/(i+j+3); 
end; end 


Next we compute the àn„’s and corresponding eigenfunctions: [v,d] = eig(A,-B). 
Table 5.7.1 gives L?An for several values of N. 

Once we found the eigenvalues and eigenvectors, we now compute the matrices D, E, 
and P. For convenience we have reordered the eigenvalues so that their numerical value 
increases with n. Furthermore, we have set an», equal to one for n = 1,2,...,N. 


[lambda,ix] = sort(temp) ; 
for i= 1:N 
for j= 1:N 

D(i,j) = vGj,ixG)); 


Green’s Functions 291 


Table 5.7.1: The Value of L?\,, for n = 1,2,...,.N as a Function of N. 


n Exact N=2 N=3 N=4 N=6 N=8 N= 10 
1 9.87 10.00 9.87 9.87 9.87 9.87 9.87 
2 39.48 42.00 42.00 39.50 39.48 39.48 39.48 
3 88.83 102.13 102.13 89.17 88.83 88.83 
4 157.91 200.50 159.99 157.96 157.91 
5 246.74 350.96 254.42 247.04 
6 355.31 570.53 376.47 356.65 
7 483.61 878.88 531.55 
8 631.65 1298.03 725.34 
9 799.44 1850.98 
10 986.96 2548.73 
end; end 
for i= 1:N 
denom = D(i,i); 
for j= 1:N 
D(i,j) = DCi,j) / denom; 
end; end 
E = transpose (D*B) ; 
P = inv(E); 


Having computed the matrices D and P, our final task is the computation of the 
Green’s function using Equation 5.7.35. The MATLAB code is: 


phi-i(1) = (1-xi)*xi; 


for i = 2:N 
phi_i(i) = xi*phi_i(i-1); 
end 


for ii = 1:idim 


x = (ii-1)*dx; 
phi_j(1) = (1-x)*x; 


for j = 2:N 
phi_j(j) = x*phi_j(j-1); 
end 
for n = 1:N 
for j= 1:N 
for i = 1:N 
g(ii) = g(ii) + D(n,j).*P(n,i).*phi_j(j) .*phi_i(i) 


. exp (-lambda(n) *time) ; 
end; end; end 


end 


In this code the parameter time denotes the quantity (t—7)/L?. Figure 5.7.1 compares 
this approximate Green’s function for various N against the exact solution. One of the 
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L g(x,tl§,t) 


Figure 5.7.1: Comparison of the exact Green’s function Lg(zx,t|€,7) for a one-dimensional heat equation 
given by Equation 5.7.38 (solid line) and approximate Green’s functions found by the Galerkin method for 
different values of N. Here (t — r)/L? = 0.001 and £ = 0.4. 


problems with this method is finding the inverse of the array E. As N increases, the 
accuracy of the inverse becomes poorer. 
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solve partial differential equations of heat conduction. 


Stakgold, I., 1997: Green’s Functions and Boundary Value Problems. Wiley-Interscience, 
720 pp. A systematic approach to boundary-value problems. 
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Chapter 6 
Probability 


So far in this book we presented mathematical techniques that are used to solve deter- 
ministic problems—problems in which the underlying physical processes are known exactly. 
In this and the next chapter we turn to problems in which uncertainty is key. 

Although probability theory was first developed to explain the behavior of games of 
chance,! its usefulness in the physical sciences and engineering became apparent by the late 
nineteenth century. Consider, for example, the biological process of birth and death. If b 
denotes the birth rate and d is the death rate, the size of the population P(t) at time t is 


P(t) = P(0)eO-%, (6.0.1) 


Let us examine the situation when P(0) = 1 and b = 2d so that a birth is twice as likely 
to occur as a death. Then, Equation 6.0.1 predicts exponential growth with P(t) = e“. 
But the first event may be a death, a one-in-three chance since d/(b + d) = 1/3, and this 
would result in the population immediately becoming extinct. Consequently we see that 
for small populations, chance fluctuations become important and a deterministic model is 
inadequate. 

The purpose of this and the next chapter is to introduce mathematical techniques that 
will lead to realistic models where chance plays an important role, and show under what 
conditions deterministic models will work. In this chapter we present those concepts that 
we will need in the next chapter to explain random processes. 


1 Todhunter, I., 1949: A History of the Mathematical Theory of Probability from the Time of Pascal 
to That of Laplace. Chelsea, 624 pp.; Hald, A., 1990: A History of Probability and Statistics and Their 
Applications before 1750. John Wiley & Sons, 586 pp. 
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6.1 REVIEW OF SET THEORY 


Often we must count various objects in order to compute a probability. Sets provide a 
formal method to aid in these computations. Here we review important concepts from set 
theory. 

Sets are collections of objects, such as the number of undergraduate students at a 
college. We define a set A either by naming the objects or describing the objects. For 
example, the set of natural numbers can be either enumerated: 


A= {1,2,3,4,...}, (6.1.1) 


or described: 
A = {I:I is an integer and I > 1}. (6.1.2) 


Each object in set A is called an element and each element is distinct. Furthermore, the 
ordering of the elements within the set is not important. 

Two sets are said to be equal if they contain the same elements and are written A = B. 
An element x of a set A is denoted by x € A. A set with no elements is called a empty 
or null set and denoted by Ø. On the other hand, a universal set is the set of all elements 
under consideration. 

A set B is subset of a set A, written B C A, if every element in B is also an element 
of A. For example, if A = {x : 0 < x < œ} and § = {x : —oœ0 < x < œ}, then AC S. 
We can also use this concept to define the equality of sets A and B. Equality occurs when 
ACBandBCA. 

The complement of A, written A, is the set of elements in S but not in A. For example, 
if A= {x:0<a2< oo} and S={xz:-0co < xr < œ}, then A= {x : — < z < 0}. 

Two sets can be combined together to form a new set. This union of A and B, written 
AU B, creates a new set that contains elements that belong to A and/or B. This definition 
can be extended to multiple sets A1, A2,..., An so that the union is the set of elements for 
which each element belongs to at least one of these sets. It is written 


N 
Ay U A2 U Ag U++-U An = |] Ai. (6.1.3) 


i=l 


The intersection of sets A and B, written AN B, is defined as the set of elements that 
belong to both A and B. This definition can also be extended to multiple sets A1, Ag,..., AN 
so that the intersection is the set of elements for which each element belongs to all of these 


sets. It is written 
N 


Ay MN A2N A3 NN An = { ) Ai. (6.1.4) 


i=l 


If two sets A and B have no elements in common, they are said to be disjoint and ANB = 0. 

A popular tool for visualizing set operations is the Venn diagram.” For sets A and B 
Figure 6.1.1 pictorially illustrates AU B, AN B, A, and AN B. 

With these definitions a number of results follow: A = A, AU A = S, ANA = Í, 
AU =A, AN =, AUS = S, ANS = A, 5 =O, and Ọ = S. Here S denotes the 
universal set. 

Sets obey various rules similar to those encountered in algebra. They include: 


2 Venn, J., 2008: Symbolic Logic. Kessinger, 492 pp. 
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S 


V 


Shaded area: ANB 


Shaded area: A Shadedarea: ANB 


Figure 6.1.1: Examples of Venn diagrams for various configurations of sets A and B. Note that in the 
case of the lower right diagram, B C A. 


. Commutative properties: AU B= BUA, ANB=BOQNA. 
. Associate properties: AU (BU C) =(AUB)UC, AN(BNC)=(ANB)NC. 
. Distributive properties: AN(BUC) = (ANB)U(ANC), AU(BNC) = (AUB)N(AUC). 


. De Morgan’s law: AUB = ANB. 


PWN re 


Finally we define the size of a set. Discrete sets may have a finite number of el- 
ements or countably infinite number of elements. By countably infinite we mean that 
we could in theory count the number of elements in the sets. Two simple examples are 
A = {1,2,3,4,5,6} and A = {1,4,16,64,...}. Discrete sets lie in opposition to continuous 
sets where the elements are infinite in number and cannot be counted. A simple example 
is A={x:0<24< 32}. 


Problems 


1. If B C A, use Venn diagrams to show that A = BU (BN A) and BAN (BNA) =b. Hint: 
Use the Venn diagram in the lower right frame of Figure 6.1.1. 


2. Using Venn diagrams, show that AUB = AU (AN B) and B= (AN B)U (AN B). Hint: 
For AN B, use the upper right frame from Figure 6.1.1. 


6.2 CLASSIC PROBABILITY 


All questions of probability begin with the concept of an experiment where the governing 
principle is chance. The set of all possible outcomes of a random experiment is called the 
sample space (or universal set); we shall denote it by S. An element of S is called a sample 
point. The number of elements in S can be finite as in the flipping of a coin twice, infinite 
but countable such as repeatedly tossing a coin and counting the number of heads, or infinite 
and uncountable, as measuring the lifetime of a light bulb. 
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Any subset of the sample set S is called an event. If this event contains a single point, 
then the event is elementary or simple. 


e Example 6.2.1 


Consider an experiment that consists of two steps. In the first step, a die is tossed. If 
the number of dots on the top of the die is even, a coin is flipped; if the number of dots on 
the die is odd, a ball is selected from a box containing blue and green balls. The sample 
space is S = {1B,1G, 2H, 2T, 3B, 3G, 4H, 4T,5B,5G,6H,6T}. The event A of obtaining a 
blue ball is A = {1B,3B, 5B}, of obtaining a green ball is B = {1G,3G,5G}, and obtaining 
an even number of dots when the die is tossed is C = {2H, 2T,4H,4T,6H,6T}. The simple 
event of obtaining a 1 on the die and a blue ball is D = {1B}. 


| Equally likely outcomes 


An important class of probability problems consists of those whose outcomes are equally 
likely. The expression “equally likely” is essentially an intuitive one. For example, if a coin 
is tossed it seems reasonable that the coin is just as likely to fall “heads” as to fall “tails.” 
Probability seeks to quantify this common sense. 


Consider a sample space S$ of an experiment that consists of finitely many outcomes 
that are equally likely. Then the probability of an event A is 


_ Number of points in A 


P(A) (6.2.1) 


~ Number of points in S` 


With this simple definition we are ready to do some simple problems. An important aid 
in our counting is whether we can count a particular sample only once (sampling without 
replacement) or repeatedly (sampling with replacement). The following examples illustrate 
both cases. 


e Example 6.2.2: Balls drawn from urns with replacement 


Imagine the situation where we have an urn that has k red balls and N — k black balls. 
A classic problem asks: What is the chance of two balls being drawn, one after another 
with replacement, where the first ball is red and the second one is black? 

We begin by labeling the k red balls with 1,2,3,...,& and black balls are numbered 
k+1,k+2,...,N. The possible outcomes of the experiment can be written as a 2-tuple 
(21, Z2), where zı € 1,2,3,...,N and 22 € 1,2,3,...,N. A successful outcome is a red ball 
followed by a black one; we can express this case by E = {(21, 22) : z1 = 1,2,...,k3 22 = 
k+1,k+2,...,N}. Now the total number of 2-tuples in the sample space is N? while the 
total number of 2-tuples in E is k(N — k). Therefore, the probability is 


pe) = +EP _ pa»), (6.2.2) 


where p = k/N. 
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Figure 6.2.1: The probability that a pair of individuals in a crowd of n people share the same birthday. 
e Example 6.2.3: Balls drawn from urns without replacement 


Let us redo the previous example where the same ball now cannot be chosen twice. We 
can express this mathematically by the condition z1 Æ z2. The sample space has N(N — 1) 
balls and the number of successful 2-tuples is again k(N — k). The probability is therefore 
given by 
k(N — k) kN-k N N 
N(N-1) N N N-1 N-1 


P(E) = (6.2.3) 
The restriction that we cannot replace the original ball has resulted in a higher probability. 
Why? We have reduced the number of red balls and thereby reduced the chance that we 
again selected another red ball while the situation with the black balls remains unchanged. 


e Example 6.2.4: The birthday problem? 


A classic problem in probability is: What is the chance that at least two individuals 
share the same birthday in a crowd of n people? Actually it is easier to solve the comple- 
mentary problem: What is the chance that no one in a crowd of n individuals shares the 
same birthday? 

For simplicity let us assume that there are only 365 days in the year. Each individual 
then has a birthday on one of these 365 days. Therefore, there are a total of (365)” possible 
outcomes in a given crowd. 

Consider now each individual separately. The first person has a birthday on one of 365 
days. The second person, who cannot have the same birthday, has one of the remaining 
364 days. Therefore, if A denotes the event that no two people have the same birthday and 
each outcome is equally likely, then 


n(A) — (365)(364) --- (365 — n + 1) 


RAS a= T l (6.2.4) 


To solve the original question, we note that P(A) = 1 — P(A) where P(A) denotes the 
probability that at least two individuals share the same birthday. 


3 First posed by von Mises, R., 1939: Uber Aufteilungs- und Besetzungswahrscheinlichkeiten. Rev. Fac. 
Sci. Istambul, 4, 145-163. 
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Figure 6.2.2: The graphical solution of whether two fellows can chat online between noon and T minutes 
after noon. The shaded area denotes the cases when the two will both be online whereas the rectangle gives 
the sample space. 


If n = 50, P(A) ~ 0.03 and P(A) ~ 0.97. On the other hand, if n = 23, P(A) ~ 0.493 
and P(A) ~ 0.507. Figure 6.2.1 illustrates P(A) as a function of n. Nymann* computed 
the probability that in a group of n people, at least one pair will have the same birthday 
with at least one such pair among the first k people. 


In the previous examples we counted the objects in sets A and S. Sometimes we can 
define these sets only as areas on a graph. This graphical definition of probability is 


_ Area covered by set A 


P(A) (6.2.5) 


~ Area covered by set S` 


The following example illustrates this definition. 
e Example 6.2.5 


Two friends, Joe and Dave, want to chat online but they will log on independently 
between noon and T minutes after noon. Because of their schedules, Joe can only wait 
t minutes after his log-on while Dave can only spare 7 minutes. Neither fellow can stay 
beyond T minutes after noon. What is the chance that they will chat? 

Let us denote Joe’s log-on time by tı and Dave’s log-on time by tz. Joe and Dave will 
chat if 0 < t2 — tı < t and 0 < tı — t2 < T. In Figure 6.2.2 we show the situation where 
these inequalities are both satisfied. The area of the sample space is T?. Therefore, from 
the geometrical definition of probability, the probability P(A) that they will chat is 


T? — (T — t}? /2 — (T — 7}? /2 


P(A) = = 


(6.2.6) 


So far there has been a single event that interests us and we have only had to compute 
P(A). Suppose we now have two events that we wish to follow. How are the probabilities 
P(A) and P(B) related? 


4 Nymann, J. E., 1975: Another generalization of the birthday problem. Math. Mag., 53, 111-125. 
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Shaded area: ANB 


Shaded area: ANB 


Figure 6.2.3: The Venn diagram used in the derivation of Property 5. 


Consider the case of flipping a coin. We could define event A as obtaining a head, A = 
{head}. Event B could be obtaining a tail, B = {tail}. Clearly AU B = {head, tail} = S, 
the sample space. Furthermore, AN B = Ý and A and B are mutually exclusive. We already 
know that P(A) = P(B) = 4. But what happens if AN B is not an empty set? 

From our definition of probability and previous examples, we introduce the following 
three basic axions: 


Axion 1: P(A), P(B) > 0, 
Axion 2: P(S) =1, 
Axion 3: P(AU B) = P(A) + P(B) if ANB=9. 


The first two axions are clearly true from the definition of probability and sample space. 
It is the third axion that needs some attention. Here we have two mutually exclusive events 
A and B in the sample space S. Because the number of points in AU B equals the number 
of points in A plus the number of points in B, n( AU B) = n(A)+n(B). Dividing both sides 
of this equation by the number of sample points and applying Equation 6.2.1, we obtain 
Axion 3 when AN B = 9. 

From these three axioms, the following properties can be written down: 


)=1- P(A) 


A 

0) =0 

A) < P(B) if ACB 
A) <1 

A 


ak wN 
VuDyuD 


UB)+ P(AN B) = P(A)+ P(B). 
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Figure 6.2.4: The Venn diagram that shows that A = (AN B)U(ANB). 

All of these properties follow readily from our definition of probability except for Prop- 
erty 5 and this is an important one. To prove this property from Axion 3, consider the 
Venn diagram shown in Figure 6.2.3. From this figure we see that 

AUB=AU(ANB) and B=(ANB)U(ANB). (6.2.7) 
From Axion 3, we have that 


P(AUB) = P(A) + P(AnB), (6.2.8) 


and 


P(B) = P(ANB) + P(AN B). (6.2.9) 


Eliminating P(A N B) between Equation 6.2.8 and Equation 6.2.9, we obtain Property 5. 
The following example illustrates a probability problem with two events A and B. 


e Example 6.2.6 


Consider Figure 6.2.4. From this figure, we see that A = (AN B) U (AN B). Because 
ANB and AN B are mutually exclusive, then from Axion 3 we have that 


P(A) = P(AN B) + P(ANB). (6.2.10) 


e Example 6.2.7 


A company has 400 employees. Every quarter, 100 of them are tested for drugs. The 
company’s policy is to test everyone at random, whether they have been previously tested 
or not. What is the chance that someone is not tested? 

The chance that someone will be tested is 1/4. Therefore, the chance that someone 
will not be tested is 1 — 1/4 = 3/4. 


| Permutations and combinations 


By now it should be evident that your success at computing probabilities lies in correctly 
counting the objects in a given set. Here we examine two important concepts for systemic 
counting: permutations and combinations. 
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A permutation consists of ordering n objects without any regard to their order. For 
example, the six permutations of the three letters a, b, and c are abc, acb, bac, bca, cab, and 
cba. The number of permutations equals n!. 

In a combination of given objects, we select one or more objects without regard to 
their order. There are two types of combinations: (1) n different objects, taken k at a time, 
without repetition, and (2) n different objects, taken k at a time, with repetitions. In the 
first case, the number of sets that can be made up from n objects, each set containing k 
different objects and no two sets containing exactly the same k things, equals 


! 
number of different combinations = B = abr (6.2.11) 


Using the three letters a, b, and c, there are three combinations, taken two letters at a time, 
without repetition: ab, ac, and bc. 

In the second case, the number of sets, consisting of k objects chosen from the n objects 
and each being used as often as desired, is 


(6.2.12) 


n+k-1 
k : 


number of different combinations = ( 


Returning to our example using three letters, there are six combinations with repetitions: 
ab, ac, bc, aa, bb, and cc. 


e Example 6.2.8 


An urn contains r red balls and b blue balls. If a random sample of size m is chosen, 
what is the probability that it contains exactly k red balls? 


i : b : 
If we choose a random sample of size m, we obtain ) possible outcomes. The 


number of samples that includes k red balls and m — k blue balls is ( J * K k ) . There- 


fore, the probability that a sample of size m contains exactly k red balls is 


0109] 
Cn) 


e Example 6.2.9 


A dog kennel has 50 dogs, including 5 German shepherds. (a) What is the probability of 
choosing 3 German shepherds if 10 dogs are randomly selected? (b) What is the probability 
of choosing all of the German shepherds in a group of 10 dogs that is chosen at random? 

Let S denote the sample space of groups of 10 dogs. The number of those groups 
is n(S) = 50!/(10!40!). Let A; denote the set of 10 dogs that contain i German shep- 
herds. Then the number of groups of 10 dogs that contain i German shepherds is n(A;) = 
10!/[i!(10 — 2)!]. Therefore, the probability that out of 50 dogs, we can select at random 10 
dogs that include ¢ German shepherds is 

n(Aj;) 10!10!40! 


PUA) = (8), ~ 410 a) sO! Coa 
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Thus, P(A3) = 1.1682 x 1078 and P(A) = 2.453 x 1078. 


e Example 6.2.10 


Consider an urn with n red balls and n blue balls inside. Let R = {r1,r2,..., rn} and 
B = {b1,b2,..., bn}. Then the number of subsets of RU B with n elements is = . On 


the other hand, any subset of RU B with n elements can be written as the union of a subset 
of R with i elements and a subset of B with n — i elements for some 0 < i < n. Because, 


for each i, there are fas such subsets, the total number of subsets of red and 
blue balls with n elements equals Yay J Go a Since both approaches must be 


equivalent, 


nj i n—=ij 
i=0 i 
because ( s ) = (") F 
n—i i 
| Conditional probability 


Often we are interested in the probability of an event A provided event B occurs. 
Denoting this conditional probability by P(A|B), its probability is given by 


3 o (6.2.14) 


pijp = PAND) 


-PO P(B) >0, (6.2.15) 


where P(AN B) is the joint probability of A and B. Similarly, 


P(BIA) = as P(A) > 0. (6.2.16) 
Therefore, 
P(AN B) = P(A|B)P(B) = P(B\A)P(A), (6.2.17) 
and we obtain the famous Bayes’ rule 
p(ajp) = EBARA (6.2.18) 


P(B) 


e Example 6.2.11 


Consider a box containing 10 pencils. Three of the pencils are defective with broken 
lead. If we draw 2 pencils out at random, what is the chance that we will have selected 
nondefective pencils? 

There are two possible ways of selecting our two pencils: with and without replacement. 
Let Event A be that the first pencil is not defective and Event B be that the second pencil 
is not defective. Regardless of whether we replace the first pencil or not, P(A) = 5 because 
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each pencil is equally likely to be picked. If we then replace the first pencil, we have the 
same situation before any selection was made and P(B|A) = P(A) = i. Therefore, 


P(AN B) = P(A)P(BIA) = 0.49. (6.2.19) 


On the other hand, if we do not replace the first selected pencil, P(B|A) = S because 
there is one fewer nondefective pencils. Consequently, 


P(AN B) = P(A)P(BIA) = — x 2 = — < 0.49. (6.2.20) 


Why do we have a better chance of obtaining defective pencils if we don’t replace the 
first one? Our removal of that first, nondefective pencil has reduced the uncertainty because 
we know that there are relatively more defective pencils in the remaining 9 pencils. This 
reduction in uncertainty must be reflected in a reduction in the chances that both selected 
pencils will be nondefective. 


| Law of total probability 


Conditional probabilities are useful because they allow us to simplify probability cal- 
culations. Suppose we have n mutually exclusive events A1, A2, ...,An whose probabilities 
sum to unity, then 


P(B) = P(B|A1)P(A1) + P(B|A2)P(A2) +--+ + P(B|An)P(An), (6.2.21) 


where B is an arbitrary event, and P(B|A;) is the conditional probability of B assuming 
Ai. In other words, the law (or formula) of total probability expresses the total probability 
of an outcome that can be realized via several distinct events. 


e Example 6.2.12 


There are three boxes, each containing a different number of light bulbs. The first box 
has 10 bulbs, of which 4 are dead. The second has 6 bulbs, of which one is dead. Finally, 
there is a third box of eight bulbs, of which 3 bulbs are dead. What is the probability of 
choosing a dead bulb if a bulb is randomly chosen from one of the three boxes? 

The probability of choosing a dead bulb is 


P(D) = P(D|B,)P(B1) + P(D|B2) P(B2) + P(D|B3)P(B3) (6.2.22) 
T (5) (5) | G) (3) | (5) G) 7 — (6.2.23) 


If we had only one box with a total 24 bulbs, of which 8 were dead, then our chance of 
choosing a dead bulb would be 1/3 > 113/360. 


| Independent events 


If events A and B satisfy the equation 


P(ANB) = P(A)P(B), (6.2.24) 
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they are called independent events. From Equation 6.2.15 and Equation 6.2.16, we see that 
if Equation 6.2.24 holds, then 


P(A|B)= P(A),  P(B|A) = P(B), (6.2.25) 


assuming that P(A) 4 0 and P(B) 4 0. Therefore, the term “independent” refers to the 
fact that the probability of A does not depend on the occurrence or non-occurrence of B, 
and vice versa. 


e Example 6.2.13 


Imagine some activity where you get two chances to be successful (for example, jumping 
for fruit still on a tree or shooting basketballs). If each attempt is independent and the 
probability of success 0.6 is the same for each trial, what is the probability of success after 
(at most) two tries? 

There are two ways of achieving success. We can be successful in the first attempt 
with P(S,) = 0.6 or we can fail and then be successful on the second attempt: P(F\ N 
S2) = P(F\)P(S2) = (0.4)(0.6) = 0.24, since each attempt is independent. Therefore, 
the probability of achieving success in two tries is 0.6 + 0.24 = 0.84. Alternatively, we 
can compute the probability of failure in two attempts: P(Fı N F2) = 0.16. Then the 
probability of success with two tries would be the complement of the probability of two 
failures: 1 — 0.16 = 0.84. 


e Example 6.2.14 


Consider the tossing of a fair die. Let event A denote the tossing of a 2 or 3. Then 


P(A) = P({2,3}) = 3. Let event B denote tossing an odd number, B = {1,3,5}. Then 
P(B) =3 


5: 
Now AN B = {3} and P(AN B) = 4. Because P(A N B) = P(A)P(B), events A and 
B are independent. 


Often we can characterize each outcome of an experiment consisting of n experiments 
as either a “success” or a “failure.” If the probability of each individual success is p, 
then the probability of k successes and n — k failures is p*(1 — p)"~*. Because there are 
n!/[k!(n — k)!] ways of achieving these k successes, the probability of an event having k 
successes in n independent trials is 


n! 


Pa (k) = map” —p)"* 


(6.2.26) 


where p is the probability of a success during one of the independent trials. 
e Example 6.2.15 


What is the probability of having two boys in a four-child family? 
Let us assume that the probability of having a male is 0.5. Taking the birth of one 


child as a single trial, 
4 anaes: 
Py(2) = ——[-]} ==. 6.2.27 
12) = zl (5) 8 ( ) 


Note that this is not 0.5, as one might initially guess. 
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Problems 
1. For the following experiments, describe the sample space: 
(a) flipping a coin twice 
(b) selecting two items out of three items {a,b,c} without replacement 
(c) selecting two items out of three items {a,b,c} with replacement 


(d) selecting three balls, one by one, from a box that contains four blue balls and five green 
balls without replacement 


(e) selecting three balls, one by one, from a box that contains four blue balls and five green 
balls with replacement. 


2. Consider two fair dice. What is the probability of throwing them so that the dots sum 
to seven? 


3. In throwing a fair die, what is the probability of obtaining a one or two on the top side 
of the cube? 


4. What is the probability of getting heads exactly (a) twice or (b) thrice if you flip a fair 
coin 6 times? 


5. An urn contains six red balls, three blue balls, and two green balls. Two balls are 
randomly selected. What is the sample space for this experiment? Let X denote the 
number of green balls selected. What are the possible values of X? Calculate P(X = 1). 


6. Consider an urn with 30 blue balls and 50 red balls in it. These balls are identical except 
for their color. If they are well mixed and you draw 3 balls without replacement, what is 
the probability that the balls are all of the same color? 


7. A deck of cards has 52 cards, including 4 jacks and 4 ten’s. What is the probability of 
selecting a jack or ten? 


8. Two boys and two girls take their place on a stage to receive an award. What is the 
probability that the boys take the two end seats? 


9. A lottery consists of posting a 3-digit number given by selecting 3 balls from 10 balls, 
each ball having the number from 1 to 10. The balls are not replaced after they are drawn. 
What are your chances of winning the lottery if the order does not matter? What are your 
chances of winning the lottery if the order does matter? Write a short MATLAB code and 
verify your results. You may want to read about the MATLAB intrinsic function randpern. 


10. A circle of radius 1 is inscribed in a square with sides of length 2. A point is selected 
at random in the square in such a manner that all the subsets of equal area of the square 
are equally likely to contain the point. What is the probability that it is inside the circle? 


11. In a rural high school, 20% of the students play football and 10% of them play football 
and wrestle. If Ed, a randomly selected student of this high school, played football, what 
is the probability that he also wrestles for his high school? 
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12. You have a well-shuffled card deck. What is the probability the second card in the deck 
is an ace? 


13. We have two urns: One has 4 red balls and 6 green balls, the other has 6 red and 4 
green. We toss a fair coin. If heads, we pick a random ball from the first urn, if tails from 
the second. What is the probability of getting a red ball? How do your results compare 
with the probability of getting a red ball if all of the red and green balls had been placed 
into a single urn? 


14. A customer decides between two dinners: a “cheap” one and an “expensive” one. The 
probability that the customer chooses the expensive meal is P(E) = 0.2. A customer who 
chooses the expensive meal likes it with a 80% probability P(L|E) = 0.8. A customer who 
chooses the cheap meal dislikes it with 70% probability P(D|C) = 0.7. 


(a) Compute the probability that a customer (1) will choose a cheap meal, (2) will be 
disappointed with an expensive meal, and (3) will like the cheap meal. 


(b) Use the law of total probability to compute the probability that a customer will be 
disappointed. 


(c) If a customer found his dinner to his liking, what is the probability that he or she chose 
the expensive meal? Hint: Use Bayes’ theorem. 


15. Suppose that two points are randomly and independently selected from the interval 
(0,1). What is the probability the first one is greater than 1/4, and the second one is less 
than 3/4? Check your result using rand in MATLAB. 


16. A certain brand of electronics chip is found to fail prematurely in 1% of all cases. If 
three of these chips are used in three independent sets of equipment, what is the probability 
that (a) all three will fail prematurely, (b) that two will fail prematurely, (c) that one will 
fail prematurely, and (d) that none will fail? 


Project: Experimenting with MATLAB’s Intrinsic Function rand 


The MATLAB function rand can be used in simulations where sampling occurs with 
replacement. If we write X = rand(1,100), the vector X contains 100 elements whose values 
vary between 0 and 1. Therefore, if you wish to simulate a fair die, then we can set up the 
following table: 


O< X< 1/6 die with one dot showing 
1/6 < X < 1/3 die with two dots showing 
1/3 < X < 1/2 die with three dots showing 
1/2 < X < 2/3 die with four dots showing 
2/3 < X < 5/6 die with five dots showing 
5/6 <X<1 die with six dots showing. 


We can then write MATLAB code that counts the number of times that we obtain a one or 
two. Call this number n. Then the probability that we would obtain one or two dots on a 
fair die is n/100. Carry out this experiment and compare your answer with the result from 
Problem 2. What occurs as you do more and more experiments? 


Probability 307 


Table 6.2.1: The Probability of a Male (Female) Freshman Having Always Had New Male 
(Female) Roommates from a Pool of m Other Male (Female) Freshmen after n Random 
Reassignments during His (Her) Freshman Year. The Numerator Is the Probability for a 
Two-Person Room; the Denominator Is the Probability for a Three-Person Room. 


Total Number of Freshmen 


n 6 12 18 24 30 36 42 48 


0.8000 0.9091 0.9412 0.9565 0.9655 0.9714 0.9756 0.9787 


j 0.3000 0.6545 0.7721 0.8300 0.8645 0.8874 0.9037 0.9158 
4 0.1920 0.5409 0.6839 0.7594 0.8059 0.8374 0.8601 0.8773 
0.0000 0.4550 0.1792 0.3015 0.3981 0.4729 0.5325 0.5801 
6 0.0000 0.1878 0.3692 0.4910 0.5750 0.6357 0.6815 0.7172 
0.0000 0.0000 0.0073 0.0385 0.0867 0.1407 0.1943 0.2450 
8 0.0000 0.0310 0.1405 0.2524 0.3459 0.4214 0.4825 0.5325 


0.0000 0.0000 0.0000 0.0012 0.0075 0.0212 0.0411 0.0658 


Project: Experimenting with 
MATLAB’s Intrinsic Function randperm 


MATLAB’s intrinsic function randperm(m) creates a random ordering of the numbers 
from 1 to m. If you execute perm = randperm(365), this would produce a vector of length 
365 and each element has a value lying between 1 and 365. If you repeat the process, you 
would obtain another list of 365 numbers but they would be in a different order. 

Let us simulate the birthday problem. Invoking the randperm command, use the first 
element to simulate the birthday of student 1 in a class of N students. Repeatedly invoking 
this command, create vector birthdays that contains the birthdays of the N students. Then 
find out if any of the days are duplicates of another. (Hint: You might want to explore the 
MATLAB command unique.) Repeating this experiment many times, compute the chance 
that a class of size N has at least two students that have the same birthday. Compare your 
results with Equation 6.2.4. What occurs as the number of experiments increases? 


Project: The Roommate Problem 


You are a freshman at a small all-male (all-female) college with m other freshmen. For 
esprit de corps, the administration requires that n times during your freshman year, you 
are randomly (with equal probability) assigned new roommates. The administration does 
not, however, require that you have never roomed with any of them previously. 

(a) Assuming that there are 2 freshmen per room (so that m + 1 is even), what is the 
probability that all of your roommates during the year have never roomed with you before? 
Verify your answer by writing a MATLAB script that simulates this housing practice. I used 
the MATLAB intrinsic functions randi(m,1,n), unique and length and ran the simulation 
10 million times. 

(b) Assuming that there are 3 freshmen per room (so that m + 1 is a multiple of 3), 
what is the probability that all of your roommates during the year have never roomed with 
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you before? Verify your answer by writing a MATLAB script that simulates this housing 
practice. I used the MATLAB intrinsic functions randperm, unique and length and ran the 
simulation 10 million times. 


6.3 DISCRETE RANDOM VARIABLES 


In the previous section we presented the basic concepts of probability. In high school 
algebra you were introduced to the concept of a variable—a quantity that could vary unlike 
constants and parameters. Here we extend this idea to situations where the variations are 
due to randomness. 

A random variable is a single-valued real function that assigns a real number, the value, 
to each sample point t of S. The variable can be discrete, such as the flipping of a coin, 
or continuous, such as the lifetime of a light bulb. The sample space S is the domain of 
the random variable X(t), and the collection of all numbers X(t) is the range. Two or 
more sample points can give the same value of X(t), but we will never allow two different 
numbers in the range of X(t) for a given t. 

The term “random variable” is probably a poor one. Consider the simple example of 
tossing a coin. A random variable that describes this experiment is 


1, sı = head, 

Al T s2 = tail. (RE 
An obvious question is: What is random about Equation 6.3.1? If a head is tossed, we 
obtain the answer one; if a tail is tossed, we obtain a zero. Everything is well defined; 
there is no element of chance here. The randomness arises from the tossing of the coin. 
Until the experiment (tossing of the coin) is performed, we do not know the outcome of 
the experiment and the value of the random variable. Therefore, a random variable is a 
variable that may take different values if a random experiment is conducted and its value is 
not known in advance. 

We begin our study of random variables by focusing on those arising from discrete 
events. If X is discrete, X assumes only finitely many or countably many values: £1, £2, £3, 

. For each possible value of x;, there is a corresponding positive probability px [ai] = 
P(X = zı), px|z2] = P(X = 22),... given by the probability mass function. For values of 
x different from «;, say 1 < x < £2, the probability mass function equals zero. Therefore, 
we have that Pi, PELA 

px[z:] = { 0, otherwise, (6.3.2) 
where i = 1,2,3,.... 

At this point it is convenient to introduce several special classes or types of random 
variables. First we have independent random variables where the realization of one does 
not affect the probability distribution of the other. Of equal importance are identically 
distributed random variables where the random variables have the same probability dis- 
tribution. Finally we can combine both properties into independent identically distributed 
(i.i.d.) random variables. This last class occurs repeatedly in common applications. 


e Example 6.3.1 


Consider a fair die. We can describe the results from rolling this fair die via the discrete 
random variable X, which has the possible values x; = 1,2,3,4,5,6 with the probability 
px [ai] = é each. Note that 0 < px[z;] < 1 here. Furthermore, 


6 
do pxiei) = 1. (6.3.3) 
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Figure 6.3.1: The probability mass function for a fair die. 
Figure 6.3.1 illustrates the probability mass function. 
e Example 6.3.2 
Let us now modify Example 6.3.1 so that 
1, Si = 1, 2, 
X[s;] = i 2, Si = 3,4, (6.3.4) 
3, Si = 5,6. 
The probability mass function becomes 
px(1] = px [2] = px[3] = 5. (6.3.5) 


e Example 6.3.3 


Consider the probability mass function: 


S R(1/2)", T= a 
px[zn] = { 0, otherwise. 


Let us (a) find the value of k, (b) find P(X = 2), (c) find P(X < 2), and (d) P(X > 1). 
From the properties of probability mass function, 


co 1 n 1 
ky (5) = hy = 2k=1. (6.3.7) 
n=0 2 


Therefore, k = 4. Note that 0 < px[zn] < 1. 
Having found k, we immediately have 


(6.3.6) 


P(X =2) = pxles] =}, (6.3.8) 


P(X < 2) = px [xo] + px [21] + px [x2] = 4, (6.3.9) 


and 


P(X >1)=1-P(X =0)=}. (6.3.10) 
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Some Properties of the Probability Mass Function px [xj] 


Having introduced the probability mass function, an alternative means of describing the 
probabilities of a discrete random variable is the cumulative distribution function. It is 
defined as 

Fx (x) = P(X < 2), —00 < £ < 00. (6.3.11) 


It is computed via 


Fy(z)= X` pxl] = X pi (6.3.12) 


zi lx visu 


Consequently, combining Equation 6.3.11 and Equation 6.3.12, we obtain 


Po<e< 0) = X p (6.3.13) 


Equation 6.3.13 gives the probability over the interval (a, b]. 
e Example 6.3.4 


A Bernoulli experiment is a random experiment, the outcome of which is a success 
or failure. Consider now a sequence of independent Bernoulli trials with probability p of 
success from trial to trial. This sequence is observed until the first success occurs. Let X 
denote a random variable that equals the trial number on which the first success occurs. 
The probability mass function is then 

px(tn] = (1 — p)” tp, a A (6.3.14) 
Let us compute the cumulative distribution function. 
For geometric series, we begin by noting that 


a 


n n-1 
= = 1. 6.3.15 
2e 2a E |r| < ( ) 


Next we check Equation 6.3.14 and determine whether it is a valid probability mass function. 
It is because 


n—-1, P = 
2 Px [en] =X (1-p)" p= aa 1, (6.3.16) 
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Figure 6.3.2: The cumulative distribution function for a Bernoulli random variable. 


where we used Equation 6.3.15. Next, we note that 


Co 


P(X>m)= Ý (1-p)" p= epee =(1- p)”. (6.3.17) 
n=m+1 
Therefore, 
Fx(x)= P(X <m)=1- P(X >m)=1-(1-p)”, (6.3.18) 


where m = 1,2,3,.... 
e Example 6.3.5: Generating discrete random variables via MATLAB 


In this example we show how to generate a discrete random variable using MATLAB’s 
intrinsic function rand. This MATLAB command produces random, uniformly distributed 
(equally probable) reals over the interval (0,1). How can we use this function, when in the 
case of discrete random variables, we have only integer values, such as k = 1,2, 3,4,5,6, in 
the case of tossing a die?” 

Consider the Bernoulli random variable X = k, k = 0,1. As you will show in your 
homework, it has the cumulative distribution function of 


0, x <0, 
F(x) = l 1—p, 0<z<]l, (6.3.19) 


See Figure 6.3.2. 

Imagine now a program that includes the MATLAB function rand, which yields the 
value t. Then, if 0 < t < 1 — p, Figure 6.3.2 gives us that X = 0. On the other hand, if 
l1—p<t< 1, then X = 1. Thus, to obtain M realizations of the Bernoulli random variable 
X, the MATLAB code would read for a given p: 


clear; 


5 This technique is known as the inverse transform sampling method. See pages 85-102 in Devroye, L., 
1986: Non-Uniform Random Variable Generation. Springer-Verlag, 843 pp. 
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Figure 6.3.3: (a) Outcomes of the Bernoulli random variable generated by the MATLAB function rand. (b) 
The computed value of the probability mass function px[1] as a function of M realization of the Bernoulli 
random variable. The dashed line is the line for the exact answer p = 0.4. 


for i= 1:M 
t = rand(1,1); 
if (t <= 1-p) X(i,1) = 0; 
else 
X(i,1) = 1; 
end; end 


The end product of this code creates a vector X of length M consisting of a random variable 
with either zeros or ones. This is shown in Figure 6.3.3(a) when p = 0.4. 

Once we have generated this random variable, we can use its relative frequency to 
compute its probability mass function and cumulative distribution function from 


Number of outcomes equal to k 


px [xr] = M ; (6.3.20) 


and 
Number of outcomes < x 


Êx (z) z M 
In Figure 6.3.3(b) we have computed the value of px[1]. Clearly it should equal p. As this 
figure shows, we obtain poor results when M is small, with 6x [1] moving randomly above 
and below the correct answer. As M becomes larger, our estimate improves. 


(6.3.21) 


Problems 
1. The Bernoulli distribution has the probability mass function 
px [zx] = P(X =k)=p*(1—p)'*, k=0,1, 


where 0 < p < 1. (a) Show that this distribution is a valid probability mass function. (b) 
Find its cumulative distribution function. 
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Px(x) 


ay 


ad X - _ + 


dx 


Figure 6.4.1: A probability density function. 


2. An experiment is performed where a digit, ranging from 0 to 9, is repeatedly and 
randomly chosen. If X denotes the times that this experiment must be repeated until the 
digit 0 is selected, find P(X). 


3. A scientific company needs a programmer who knows an unusual programming language. 
If only 5% of programmers know this language, how many programmers should the company 
interview to have a 75% chance of finding such a programmer? 


6.4 CONTINUOUS RANDOM VARIABLES 


In the previous section we examined random variables that can assume only certain 
discrete values. Here we extend the concept of random variables so that they can take on 
values over a continuous interval. Typical examples of continuous random variables include 
the noisy portion of the voltage within an amplifier, the phase of a propagating wave, and 
the amount of precipitation. 

An important quantity that we introduced in the previous section was the probability 
mass function. What is the corresponding function for continuous random variables? From 
the fundamental concepts of probability, we know that the probability of a continuous 
variable assuming one specific value out of its possible range values equals zero; it is merely 
one point out of an infinite number of points in the sample space. On the other hand, there 
is a finite probability that the value assumed by the random variable X will lie within an 
arbitrarily small interval dz and this probability will depend on the length of the interval. 

Another factor that should influence the probability is the value of x. There is no 
reason why the probability of X should be independent of x. Consequently, an equation 
for probability in the interval x < X < x + dz requires a function px (x), which acts as a 
weighting function and models the relative frequency behavior of X. For these reasons, the 
probability that a continuous random variable X will assume a value lying between x and 
x + dz is given by 

P(x < X < zx + dz) = px (x) dz. (6.4.1) 


Figure 6.4.1 illustrates a possible example of px (x) where the shaded area equals the prob- 
ability P(x < X < x + dz). Clearly the function px(x) = P(x < X < x + dz)/dz has the 
dimension of probability per infinitesimal interval dx and is called, for that reason, the prob- 
ability density. Furthermore, although px(x) dz < 1, this does not mean that px (a) < 1. 
A family of random variables having the same probability density is identically distributed. 

The function px (x) must also satisfy several additional conditions. Because probability 
cannot be negative, px(a) > 0 of all x. Furthermore, as Figure 6.4.1 suggests, if we add 


314 Advanced Engineering Mathematics: A Second Course 


Some Properties of the Probability Density Function px (x) 


up all of the possible values of x, then we have a certain event. We can express this 
mathematically by 4 
j px(x)dx =1. (6.4.2) 
—oo 
Thus, a probability density has the properties given by Equation 6.4.1 and Equation 6.4.2. 
It must also be a single-valued function of x. Note that these conditions do not require that 
px (a) is a continuous function of z. 

Let us now consider the probability P(a < X < b) where a and b are constants. If 
we subdivide the range of x between a and b into infinitesimal intervals (7,x + dx), the 
probability that the random variable will assume a value from one such interval is given by 
Equation 6.4.1. The probability that the variable will assume a value in the interval (a, b) 
equals the sum of the probabilities from each subinterval between a and b and is given by 
the area under the curve p(x) between x = a and x = b. Therefore, 


b 


P(a< X <b)= / px (x) da. (6.4.3) 
If a = —onv, we have that 
P(X <b) = f px (x) da. (6.4.4) 
Alternatively, setting b = oo, ü 
P(a< X) = a px (x) da. (6.4.5) 


From Equation 6.4.3 we also have 
P(X >a) =1- P(X <a)=1- | px(a)de = | px (ax) da. (6.4.6) 


From Equation 6.4.4 we now define 


x 


Fx) = P(X <2) =f px(6)ab. (6.4.7) 


—oCo 


This function Fy (x) is called the cumulative distribution function, or simply the distribution 
function, of the random variable X. Clearly, 


px (x) = Fx(z). (6.4.8) 


Therefore, from the properties of px (x), we have that (1) Fy (x) is a nondecreasing function 
of xz, (2) Fx(—oo) = 0, (3) Fx (oo) = 1, and (4) P(a < X < b) = Fx(b) — Fx (a). 


Probability 315 


e Example 6.4.1 


The continuous random variable X has the probability density function 


k (a = z’) 0<a<1 
= 7 i .4.9 
px (2) { 0, otherwise. cea?) 
What must be the value of k? What is the cumulative distribution function? What is 
P(X < 1/2)? 
From Equation 6.4.2, we have that 


Ex l z? | k 
J px(a)de =k f (a — 2”) dz =k—- =| ==. (6.4.10) 
aes 0 2 3}, 6 
Therefore, k must equal 6. 
Next, we note that 
Fx (x) = P(X < x)= / px (€) dé. (6.4.11) 
Ifa <0, Fx(x) =0. For 0 <a <1, then 
ER ee ee ee ee ee 
x(t) =6] (€-€7) d€=6(~-—~}] =3a7 - 22°. (6.4.12) 
0 2 3 /lo 
Finally, if x > 1, 
1 
Fx (x) = J (E - £) d€=1. (6.4.13) 
0 
In summary, 
0, 0<a, 
Fx (x) = 3x? — 27°, 0<z<1, (6.4.14) 
1, 1l<2 


e Example 6.4.2: Generating continuous random variables via MATLAB® 


In the previous section we showed how the MATLAB function rand can be used to gen- 
erate outcomes for a discrete random variable. Similar considerations hold for a continuous 
random variable. 

Consider the exponential random variable X. Its probability density function is 


0, x <0, 
px (x) = { de", OMe, (6.4.15) 


where à > 0. For homework you will show that the corresponding cumulative distribution 


function is 
0, x <0, 
Fx(x) = { ia ee ek (6.4.16) 


6 This technique is known as the inverse transform sampling method. See pages 27-39 in Devroye, L., 
1986: Non-Uniform Random Variable Generation. Springer-Verlag, 843 pp. 
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Figure 6.4.2: The cumulative distribution function for an exponential random variable. 


Figure 6.4.2 illustrates this cumulative density function when A = 1. How can we use these 
results to generate a MATLAB code that produces an exponential random variable? 

Recall that both MATLAB function rand and the cumulative distribution function pro- 
duce values that vary between 0 and 1. Given a value from rand, we can compute the 
corresponding X = x, which would give the same value from the cumulative distribution 
function. In short, we are creating random values for the cumulative distribution function 
and using those values to give the exponential random variable via 


X = x = —ln(1 — rand) /A, (6.4.17) 


where we have set F(x) = rand. Therefore, the MATLAB code to generate exponential 
random variables for a particular lambda is 


clear; 
for i = 1:M 

t = rand(1,1); 

X(i,1) = -log(1-t) / lambda; 
end 


where M is the number of experiments that we run. In Figure 6.4.3(a) we illustrate the first 
200 outcomes from our numerical experiment to generate an exponential random variable. 

To compute the probability density function we use the finite difference approximation 
of Equation 6.4.1, or 


Number of outcomes in [zo — Aa/2, xo + Ax/2] 
MAz í 


p(ao) = (6.4.18) 
where Az is the size of the bins into which we collect the various outcomes. Figure 6.4.3(b) 
illustrates this numerical estimation of the probability density function in the case of an 
exponential random variable. The function fx (x) was created from the MATLAB code: 


clear; 

delta_x = 0.2; lambda = 1; M = 1000; % Initialize Az, À and M 
% sample M outcomes from the uniformly distributed distribution 
t = rand(M,1); 

% generate the exponential random variable 

x = - log(1-t)/lambda; 
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Figure 6.4.3: (a) Outcomes of a numerical experiment to generate an exponential random variable using 
the MATLAB function rand. (b) The function x(x) given by Equation 6.4.18 as a function of x for an 
exponential random variable with M = 1000. The dashed black line is the exact probability density function. 


% create the various bins [zo — Azx/2, xo + Ax/2] 
bincenters=[delta_x/2:delta_x:5]; 
bins=length(bincenters); % count the number of bins 

% now bin the M outcomes into the various bins 

[n,x out] = hist(x,bincenters) ; 

n = n / (delta-x*M); % compute the probability per bin 
bar_h = bar(x_out,n); % create the bar graph 

bar_child = get (bar_h,’Children’) ; 

set (bar_child, ’CData’,n); 

colormap (Autumn) ; 


Problems 


1. The probability density function for the exponential random variable is 


with A > 0. Find its cumulative distribution function. 
2. Given the probability density function 


g= ka, 0<2“<2, 
a = 1 0, otherwise, 


318 Advanced Engineering Mathematics: A Second Course 


probability density 


0.35 0.4 0.45 0.5 0.55 0.6 0.65 


Figure 6.4.4: Computed probability density function for the sum S = (X1 + X2 + X3 + --- + Xio0)/100, 
where X; is the ith sample from a uniform distribution. 


where k is a constant, (a) compute the value of k, (b) find the cumulative density function 
Fx (a), and (c) find the P(1 < X < 2). 


3. Given the probability density function 


_ | eih= eps |z| <1, 
px(2) = { 0, le| > 1, 


where k is a constant, (a) compute the value of k and (b) find the cumulative density 
function Fy (x). 


Project: Central Limit Theorem 


Consider the sum S = (Xı + X2 + X3 +---+ Xı00)/100, where X; is the ith sample 
from a uniform distribution. 


Step 1: Write a MATLAB program to compute the probability density function of S. See 
Figure 6.4.4. 


Step 2: The central limit theorem states the distribution of the sum (or average) of a 
large number of independent, identically distributed random variables will be approximately 
normal, regardless of the underlying distribution. Do your numerical results agree with this 
theorem? 


6.5 MEAN AND VARIANCE 


In the previous two sections we explored the concepts of the random variable and 
distribution. Here we introduce two parameters, mean and variance, that are useful in 
characterizing a distribution. 
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The mean ux is defined by 


5 Tk px|zk];, X discrete, 
ux =E(X)=< „%& (6.5.1) 
/ xpx(x)dx, X continuous. 


The mean provides the position of the center of the distribution. The operator E(X), which 
is called the expectation of X , gives the average value of X that one should expect after many 
trials. 

Two important properties involve the expectation of the sum and product of two ran- 
dom variables X and Y. The first one is 


E(X +Y) = E(X)+ E(Y). (6.5.2) 
Second, if X and Y are independent random variables, then 
E(XY) = E(X)E(Y),. (6.5.3) 


The proofs can be found elsewhere.” 
The variance provides the spread of a distribution. It is computed via 


oxy = Var(X) = E{[X — E(X)}*}, (6.5.4) 
or 
Sek — ux)°px[£k], X discrete, 
=e OS (6.5.5) 
i. (a — wx)*px(x)dx, X continuous. 


If we expand the right side of Equation 6.5.4, an alternative method for finding the variance 
is 
oy = Var(X) = E(X*) — [E(X)/’, (6.5.6) 


where 
5 £p px|xk], X discrete, 
k 


E(X”) = A (6.5.7) 
jf x”px(x)dx, X continuous. 
e Example 6.5.1: Mean and variance of M equally likely outcomes 
Consider the random variable X = k where k = 1,2,...,M. If each event has an 


equally likely outcome, px[x;,] = 1/M. Then the expected or average or mean value is 
a M(M+1) M+1 


6.5.8 
2M 2 (a 


T For example, Kay, S. M., 2006: Intuitive Probability and Random Processes Using MATLAB. Springer, 
833 pp. See Sections 7.7 and 12.7. 
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Note that the mean does not equal any of the possible values of X. Therefore, the expected 
value need not equal a value that will be actually observed. 
Turning to the variance, 


Var(X) = (M +1) [(2M + 1)/6 — (M +1)/4] (6.5.9) 
= (M+1)[4M+2-3M —3]/12 (6.5.10) 
= (M +1)(M — 1)/12 = (M° — 1)/12, (6.5.11) 
because 
M 
E(X?) = - Je M(M a E See (6.5.12) 
k=1 


We used Equation 6.5.6 to compute the variance. 


e Example 6.5.2 


Let us find the mean and variance of the random variable X whose probability density 


function is 
ka, 0<a<l, 


PEO)= { 0, otherwise. (Sony) 


1 3 
k k 
ux = E(X) z a(kx)dx = ——| = =. (6.5.14) 
0 3/5 3 
From Equation 6.5.6, the variance of X is 
i k? kéj k 2 
oy = Var(X) = E(X?) — [E(X)|? =} x’ (ka) dx = = (6.5.15) 
0 9 4 |, 9 9 
e Example 6.5.3: Characteristic functions 
The characteristic function of a random variable is defined by 
x(w) = Elexp(iwX)]. (6.5.16) 
If X is a discrete random variable, then 
ox(w) = X. px[zale*. (6.5.17) 
k=- 
On the other hand, if X is a continuous random variable, 
ox(w) = / px(x)e* da, (6.5.18) 


the inverse Fourier transform (times 27) of the Fourier transform, px (x). 
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Characteristic functions are useful for computing various moments of a random variable 
via 
1 d” w 


6.5.19 
a” dw” ( ) 


w=0 
This follows by taking repeated differentiation of Equation 6.5.16 and then evaluating the 
differentiation at w = 0. 

Consider, for example, the exponential probability density function px(x) = ’e~** 
with z, à > 0. A straightforward calculation gives 


ex (w) = ay (6.5.20) 
Substituting Equation 6.5.20 into Equation 6.5.19 yields 
E(X”) = 2 (6.5.21) 
In particular, 
E(X) = k and E(X’) = =. (6.5.22) 


Consequently, wx = 1/A and Var(X) = E(X?) — på = 1/A?. 
e Example 6.5.4: Characteristic function for a Gaussian distribution 


Let us find the characteristic function for the Gaussian distribution and then use that 
characteristic function to compute the mean and variance. 


Because 1 
2 2 
px (a) = oo” LREN, (6.5.23) 
the characteristic function equals 
Lf EE 
; 1 oo aie ONO 

Z P SA J ep] is I ) | da} (6.5.25) 
= eiwH—o7w? /2 (6.5.26) 


because the quantity within the wavy brackets equals one. 
Given this characteristic function, Equation 6.5.26, we have that 


px (w) = (iu — 02w) ew? /2, (6.5.27) 
Therefore, ¢4,(0) = iu and from Equation 6.5.19, ux = E(X) = u. Furthermore, 


1 2 


klw) = (iu — o w)2etu-ow?/2 agree se, (6.5.28) 


Consequently, ¢%(0) = —y? — o? and Var(X) = E(X?) — u% = 0°. 
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e Example 6.5.5: (Weak) law of large numbers 


One of the reasons why independent identically distributed (i.i.d.) random variables 
play such a large role in probability and statistics lies in the (weak) law of large numbers. If 
X,X2,X3,...,X, denote iid. random variables and A, = + )>;_, Xj, then P(|Ay — u| > 
€) > 0, as n > oo for any € > 0. How is this law useful in daily life? Let us observe some 
random variable many times and take the average of these observations. The law of large 
numbers predicts that this average will converges to a single value, namely the mean. 


Problems 
1. Let X(s) denote a discrete random variable associated with a fair coin toss. Then 


0, s = tail, 
1; s = head. 


Find the expected value and variance of this random variable. 

2. The geometric random variable X has the probability mass function: 
px |p] = P(X =k) = p(1—p)*}, k =1,2,3,.... 

Find its mean and variance. Hint: 


co 1 co 
kr"! = : k(k = 1)? = —— —, |r| <1, 
Ln dae & =r 


and E(X?) = E[X(X —1)] + E(X). 
3. Given 


(ee ka(2— x) 0<4 <2, 
BEST 0, otherwise, 


(a) find k and (b) its mean and variance. 
4. Given the probability density 

px (a) = (a2 — x?) 3, v> -—ż4, 
find its characteristic function using integral tables. 


For the following distributions, first find their characteristic functions. Then compute the 
mean and variance using Equation 6.5.19. 


5. Binomial distribution: 
_ fr k n-k 
pxl = (2) ofa <p <1, 


where q = 1 — p. Hint: Use the binomial theorem to simplify Equation 6.5.17. 
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6. Poisson distribution: 


7. Geometric distribution: 
pxlzr] =q"p, O<p<1, 


where q = 1 — p. 
8. Uniform distribution: 


px(x) = ; b>a>0. 


Project: MATLAB’s Intrinsic Function mean and var 


MATLAB has the special commands mean and var to compute the mean and variance, 
respectively, of the random variable X. Use the MATLAB command randn to create a 
random variable X(n) of length N. Then, find the mean and variance of X(n). How do these 
parameters vary with N? 


Project: Monte Carlo Integration and Importance Sampling 


Consider the integral I = J v1 -— z? dx = 7/4. If we were to compute it numerically 
by the conventional midpoint rule, the approximate value is given by 


In = 5Y flon), (1) 


where f(z) = V1 — zr? and zn = (n — 1/2)/N. For N = 10, 50, 100, and 500, the absolute 

value of the relative error is 2.7 x 107%, 2.4 x 1074, 8.6 x 107°, and 7.7 x 107°, respectively. 
Monte Carlo integration is a simple alternative method for doing the numerical inte- 

gration using random sampling. It is a particularly powerful technique for approximating 

complicated integrals. Here you will explore a simple one-dimensional version of this scheme. 
Consider the random variable: 


M 


In = 57 YS Flam), (2) 


where £m is the mth sample point taken from the uniform distribution. Im is a random 
variable because it is a function of the random variable £m. Therefore, 


Wee tec ft 
Elu) = jg D EE) = 57 Df Feed 
A 1 1 
=5 f f(x)dx = A (x) dx = I, 


because p(x), the probability of the uniform distribution, equals 1. Furthermore, as we 
increase the number of samples M, IĮ approaches I. By the strong law of large numbers, 
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M = 100 M = 500 


a 


probability density 


-Oos —0.06 -0.04 -0.02 0 0.02 0.04 0.06 0.08 80a -0.03 -0.02 -0.01 0 0.01 0.02 0.03 0.04 
approximate — exact value of the integral approximate — exact value of the integral 


Figure 6.5.1: The probability density function arising from using Monte Carlo integration to compute 
J; V1 -— x? dx for various values of M. 


this limit is guaranteed to converge to the exact solution: P(imm— Im —I) = 1. Equation 
(2) is not the midpoint rule because the uniform grid £n has been replaced by randomly 
spaced grid points. 


Step 1: Write a MATLAB program that computes Im for various values of M when £m is 
selected from a uniform distribution. By running your code thousands of times, find the 
probability density as a function of the difference between IĮ and J. Compute the mean 
and variance of Im. How does the variance vary with M? See Figure 6.5.1. 


The reason why standard Monte Carlo integration is not particularly good is the fact 
that we used a uniform distribution. A better idea would be to sample from regions where 
the integrand is larger. This is the essence of the concept of importance sampling: That 
certain values of the input random variable £m in a simulation have more impact on the 
parameters being estimated than others. 

We begin by noting that 


rel Towe | E C pila) de, 


Pı (x 


where pı(x) is a new probability density function that replaces the uniform probability 
distribution and is relatively larger when f(x) is larger and relatively smaller when f(x) is 
smaller. 

The question now becomes how to compute pı(x). We shall use the VEGAS algo- 
rithm, which constructs pi(a) by sampling f(x) K times, where K < M. Within each kth 
subinterval we assume that there are M/K uniformly distributed points. Therefore, 


K f (sm) 
Fia F (sx) i 


where s; is the center point of the kth subinterval within which the mth point is located. 
For each m, we must find £m. This is done in two steps: First we randomly choose the kth 


Pı (Lm) = 
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Figure 6.5.2: The probability density function arising from using importance sampling with Monte Carlo 
1 
integration to compute i V1— x? dx for various values of K and M = 100. 


subinterval using a uniform distribution. Then we randomly choose the point £m within 
that subinterval using a uniform distribution. Therefore, our modified integration scheme 
becomes 


1 r 
B 3 


m=1 


Now, 


M M 1 
ByE a LL, farO 


M 24" |pi(em) Dil 
1 M 1 1 

= dz = dz =I 
mÈ j| tou= f fodar 


because pa(x) = 1. 


Step 2: Write a MATLAB program that computes Im for various values of K for a fixed value 
of M. Recall that you must first select the subdivision using the MATLAB function rand 
and then the value of £m within the subdivision using a uniform distribution. By running 
your code thousands of times, find the probability density as a function of the difference 
between Im and J. Compute the mean and variance of Im. How does the variance vary 
with M? See Figure 6.5.2. 


6.6 SOME COMMONLY USED DISTRIBUTIONS 


In the previous sections we introduced the concept of probability distributions and their 
description via mean and variance. In this section we focus on some special distributions, 
both discrete and continuous, that appear often in engineering. 
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| Bernoulli distribution 


Consider an experiment where the outcome can be classified as either a success or 
failure. The probability of a success is p and the probability of a failure is 1 — p. Then these 
“Bernoulli trials” have a random variable X associated with them where the probability 
mass function is given by 


px [ze] = P(X = k) = p" (1 — p)*-*, k=0,1, (6.6.1) 


where 0 < p < 1. From Equation 6.3.12 the cumulative density function of the Bernoulli 
random variable X is 


0, x <0, 
r= fir 0<a<1, (6.6.2) 
1 l<a. 


’ 


The mean and variance of the Bernoulli random variable X are 


ux = E(X) =p, and o% = Var(X) = p(1 — p). (6.6.3) 


e Example 6.6.1 


A simple pass and fail process is taking a final exam, which can be modeled by a 
Bernoulli distribution. Suppose a class passed a final exam with the probability of 0.75. If 
X denotes the random variable that someone passed the exam, then 


E(X)=p=0.75, and Var(X) = p(1 — p) = (0.75)(0.25) = 0.1875. (6.6.4) 


| Geometric distribution 


Consider again an experiment where we either have success with probability p or failure 
with probability 1—p. This experiment is repeated until the first success occurs. Let random 
variable X denote the trial number on which this first success occurs. Its probability mass 
function is 


px [2x] = P(X =k) = p(1— p)*},7 imal ey ee eee (6.6.5) 

From Equation 6.3.12 the cumulative density function of this geometric random variable X 
is 

Fx (x) = P(X < z)=1- (1 — p)". (6.6.6) 


The mean and variance of the geometric random variable X are 


ux = E(X) =-=, and =o = Var(X) = ——. (6.6.7) 


1 
Pp p 
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e Example 6.6.2 

A particle within an accelerator has the probability 0.01 of hitting a target material. 
(a) What is the probability that the first particle to hit the target is the 50th? (b) What is 
the probability that the target will be hit by any particle? 


P(first particle to hit is the 50th) = 0.01(0.99)*° = 0.0061. (6.6.8) 


50 
P(target hit by any of first 50th particles) = 5 0.01(0.99)”7+ = 0.3950. (6.6.9) 
n=1 


e Example 6.6.3 


The police ticket 5% of parked cars. Assuming that the cars are ticketed independently, 
find the probability of 1 ticket on a block with 7 parked cars. 
Each car is a Bernoulli trial with P(ticket) = 0.05. Therefore, 


P(1 ticket on block) = P(1 ticket in 7 trials) = @ (0.95)°(0.05) = 0.2573. (6.6.10) 


| Binomial distribution 


Consider now an experiment in which n independent Bernoulli trials are performed and 
X represents the number of successes that occur in the n trials. In this case the random 
variable X is called binomial with parameters (n, p) with a probability mass function given 
by 


where 0 < p < 1, and 
n n! 
G = k(n — k)” (6.6.12) 


the binomial coefficient. The term p! arises from the k successes while (1 — p)"~* is due 
to the failures. The binomial coefficient gives the number of ways that we pick those k 
successes from the n trials. 


The corresponding cumulative density function of X is 
_ n \ Wk n—-k 
Fx(e) = (p) a-p) ,  nsacntl. (6.6.13) 
k=0 
The mean and variance of the binomial random variable X are 


ux = E(X) = np, and ox = Var(X) = np(1 — p). (6.6.14) 
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A Bernoulli random variable is the same as a binomial random variable when the parameters 
are (1,7). 


e Example 6.6.4 


Let us find the probability of rolling the same side of a die (say, the side with N dots 
on it) at least 3 times when a fair die is rolled 4 times. 

During our 4 tosses, we could obtain no rolls with N dots on the side (k = 0), one roll 
with N dots (k = 1), two rolls with N dots (k = 2), three rolls with N dots (k = 3), or 
four rolls with N dots (k = 4). If we define A as the event of rolling a die so that the side 
with N dots appears at least three times, then we must add the probabilities for k = 3 and 
k = 4. Therefore, 


TPE TORS Q p(l- p)" + & pi(l—p)? (6.6.15) 
7 mC py + pa p)? = 0.0162 (6.6.16) 


because p = §. 


e Example 6.6.5 


If 10 random binary digits are transmitted, what is the probability that more than 
seven l’s are included among them? 
Let X denote the number of 1’s among the 10 digits. Then 


P(X > 1) = PX =8) + PX = 9) + P(X = 10) = pxles] +pxleol+pxlero] (6617) 
(8) (3) (2) t) G@) G) ta) G) G) ee 


1\'° 56 
= (45 +10 +1) (5) = Toor: (6.6.19) 


| Poisson distribution 


The Poisson probability distribution arises as an approximation for the binomial dis- 
tribution as n > oo and p > 0 such that np remains finite. To see this, let us rewrite the 
binomial distribution as follows: 


P(X =k) = n! aV j AYE n(n- 1)(n-2):- (n-k +1)" (1- Af)” 
k!(n — k)! \n n nk n! (L—A/n)*? 
(6.6.20) 
if A = np. For finite A, 
k n 
lim (1- >) => 1, lim (1 - >) +e, (6.6.21) 
n= n n= oo n 
a 1 2 k+1 
iin ED ee (6.6.22) 
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Therefore, for large n, small p and moderate A, we can approximate the binomial distribution 
by the Poisson distribution: 


k 
px |p] = P(X = k) aor, k=0,1,.... (6.6.23) 
The corresponding cumulative density function of X is 
Fx (zx) =e p n<a<ntl. (6.6.24) 
k=0 


The mean and variance of the Poisson random variable X are 
ux = E(X)=), and ay = Var(X) =). (6.6.25) 


In addition to this approximation, the Poisson distribution is the probability distribution 
for a Poisson process. But that has to wait for the next chapter. 


e Example 6.6.6 


Consider a student union on a campus. On average 3 persons enter the union per 
minute. What is the probability that, during any given minute, 3 or more persons will 
enter the union? 

To make use of Poisson’s distribution to solve this problem, we must have both a large 
n and a small p with the average \ = np = 3. Therefore, we divide time into a large number 
of small intervals so that n is large while the probability that someone will enter the union 
is small. Assuming independence of events, we have a binomial distribution with large n. 
Let A denote the event that 3 or more persons will enter the union, then 
Br er eae 


+ x = 0.423. (6.6.26) 


P(A) = px [0] + px [1] + px [2] =e? p Ia 


Therefore, P(A) = 1 — P(A) = 0.577. 


| Uniform distribution 


The continuous random variable X is called uniform if its probability density function 


_ J 1/(-a), ax<x<b, 
oC { 0, otherwise. (6620) 


is 


The corresponding cumulative density function of X is 


0, z <a, 
Fx(x) = i (a — a)/(b—a), a<a2<b, (6.6.28) 
1, b<a. 


The mean and variance of a uniform random variable X are 


ux = E(X) = (a +b), and o% = Var(X) = ( f (6.6.29) 
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Uniform distributions are used when we have no prior knowledge of the actual probability 
density function and all continuous values in some range appear equally likely. 


| Exponential distribution 


The continuous random variable X is called exponential with parameter A > 0 if its 
probability density function is 


ceo, > 0, 
px(x) = { n ae (6.6.30) 


The corresponding cumulative density function of X is 


1—e* > 
Fy(£) = { o i K z (6.6.31) 


The mean and variance of an exponential random variable X are 
ux = E(X) =1/A, and oy = Var(X) =1/)?. (6.6.32) 


This distribution has the interesting property that is “memoryless.” By memoryless, 
we mean that for a nonnegative random variable X, then 


P(X >s+t|X >t)= P(X >s), (6.6.33) 


where x,t > 0. For example, if the lifetime of a light bulb is exponentially distributed, then 
the light bulb that has been in use for some hours is as good as a new light bulb with regard 
to the amount of time remaining until it fails. 

To prove this, from Equation 6.2.4, Equation 6.6.33 becomes 


P(X >s+tand X >t) 
P(X >t) 


= P(X > 8), (6.6.34) 


or 
P(X >s+tand X >t) = P(X >t)P(X >s), (6.6.35) 


since P(X > s +t and X >t) = P(X > s+ t). Now, because 


P(X >s+t)=1-— [i-e aet] = eTA H), (6.6.36) 
P(X > 8s)=1-— (1-e7™) =e", (6.6.37) 

and 
P(X >t)=1-(l-e™)=e™. (6.6.38) 


Therefore, Equation 6.6.35 is satisfied and X is memoryless. 
e Example 6.6.7 
A component in an electrical circuit has an exponentially distributed failure time with 


a mean of 1000 hours. Calculate the time so that the probability of the time to failure is 
less than 107%. 
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Let the exponential random variable X = k have the units of hours. Then \ = 107°. 
From the definition of the cumulative density function, 


Fy (xz) = P(X < x) = 0.001, and 1 — exp(—Az;) = 0.001. (6.6.39) 
Solving for x, 


x, = —In(0.999)/d = 1. (6.6.40) 


e Example 6.6.8 


A computer contains a certain component whose time (in years) to failure is given by 
the random variable T distributed exponentially with A = 1/5. If 5 of these components 
are installed in different computers, what is the probability that at least 2 of them will still 
work at the end of 8 years? 

The probability that a component will last 8 years or longer is 


P(T > 8) = e785 = 0.2019, (6.6.41) 


because \ = 1/5. 
Let X denote the number of components functioning after 8 years. Then, 
P(X > 2) =1-— P(X =0)- P(X =1) (6.6.42) 
=1- (3) (0.2019)? (0.7981)5 — (7) (0.2019)'(0.7981)* (6.6.43) 


= 0.2666. (6.6.44) 


Normal (or Gaussian) distribution 


The normal distribution is the most important continuous distribution. It occurs in 
many applications and plays a key role in the study of random phenomena in nature. 
A random variable X is called a normal random variable if its probability density 


function is 
e~(@—H)"/ (207) 


x = ty ne | 6.6.45 

px (a) = — 3 (6.6.45) 
where the mean and variance of a normal random variable X are 

ux = E(X) =n, and oł% = Var(X) = 0°. (6.6.46) 


The distribution is symmetric with respect to x = u and its shape is sometimes called 
“bell shaped.” For small g? we obtain a high peak and steep slope while with increasing o? 
the curve becomes flatter and flatter. 

The corresponding cumulative density function of X is 


1 Z Lannion 1 @-wW/e os 
Fx (x) = = | e EH) 2 ag = = | eE /2 dé. (6.6.47) 
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The integral in Equation 6.6.46 must be evaluated numerically. It is convenient to introduce 
the probability integral: 


ey eee 
(z) = ie = 8/2 dé. (6.6.48) 
Note that ®(—z) = 1 — ®(z). Therefore, 
Fx (x) = (=) i (6.6.49) 
and 
P(a < X <b) = Fx (b) — Fx (a). (6.6.50) 


Consider now the intervals consisting of one a, two o, and three o around the mean wp. 
Then, from Equation 6.6.50, 


P(u— o< X < u+ o) = 0.68, (6.6.51) 
P(u— 20 < X < u+ 2o) = 0.955, (6.6.52) 

and 
P(u— 30 < X < u +30) = 0.997. (6.6.53) 


Therefore, approximately 2 of the values will be distributed between u — o and u +a, 


approximately 95% of the values will be distributed between — 20 and u +20, and almost 
all values will be distributed between u — 30 and u + 30. For most uses, then, all values 
will lie between p — 30 and u + 3<, the so-called “three-sigma limits.” 

As stated earlier, the mean and variance of a normal random variable X are 


ux = E(X) =n, and o% = Var(X) = 0°. (6.6.54) 


The notation N (u; o) commonly denotes that X is normal with mean pu and variance 
a”. The special case of a normal random variable Z with zero mean and unit variance, 


N (0,1), is called a standard normal random variable. 

Problems 
1. Four coins are tossed simultaneously. Find the probability function for the random 
variable X that gives the number of heads. Then compute the probabilities of (a) obtaining 
no heads, (b) exactly one head, (c) at least one head, and (d) not less than four heads. 
2. A binary source generates the digits 1 and 0 randomly with equal probability. (a) What 
is the probability that three 1’s and three 0’s will occur in a six-digit sequence? (b) What 


is the probability that at least three 1’s will occur in a six-digit sequence? 


3. Show that the probability of exactly n heads in 2n tosses of a fair coin is 


1-3-5---2n-1 


Px|®n] = 9G. On 


4. If your cell phone rings, on average, 3 times between noon and 3 P.M., what is the 
probability that during that time period you will receive (a) no calls, (b) 6 or more calls, and 
(c) not more than 2 calls? Assume that the probability is given by a Poisson distribution. 
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5. A company sells blank DVDs in packages of 10. If the probability of a defective DVD is 
0.001, (a) what is the probability that a package contains a defective DVD? (b) what is the 
probability that a package has two or more defective DVDs? 


6. A school plans to offer a course on probability in a classroom that contains 20 seats. 
From experience they know that 95% of the students who enroll actually show up. If the 
school allows 22 students to enroll before the session is closed, what is the probability of 
the class being oversubscribed? 


7. The lifetime (in hours) of a certain electronic device is a random variable T having a 
probability density function pr(t) = 100H(t—100)/t?. What is the probability that exactly 
3 of 5 such devices must be replaced within the first 150 hours of operation? Assume that 
the events that the ith device must be replaced within this time are independent. 


6.7 JOINT DISTRIBUTIONS 


In the previous sections we introduced distributions that depended upon a single ran- 
dom variable. Here we generalize these techniques for two random variables. The range of 
the two-dimensional random variable (X,Y) is Rxy = {(x,y);€ € S and X(£) = 2, Y (£) = 


y}- 


| Discrete joint distribution 


Let X and Y denote two discrete random variables defined on the same sample space 
(jointly distributed). The function pxy[z;, y;] = P[X = 2;,Y = yj] is the joint probability 
mass function of X and Y. As one might expect, pxy [xi, yj] > 0. 

Let the sets of possible values of X and Y be A and B. If x; ¢ A or y; ¢ B, then 
pxy [%i, yj] = 0. Furthermore, 


oy pxy (Xi, yj) = 1. (6.7.1) 


xiEA,y;EB 


The marginal probability functions of X and Y are defined by 


pxlz:] = D pxy |xi, yj], (6.7.2) 
yjEB 
and 
prlu] = X pxy[zi, yj). (6.7.3) 
x;EA 


If X and Y are independent random variables, then pxy|xi, y;] = px [xi]: py [y3]. 
e Example 6.7.1 
A joint probability mass function is given by 


k(x; + 2y;), Cpa 1, 253,47 = 1,2; 


pxy [zi yj] = { 0. Aric (6.7.4) 
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Let us find the value of k, px[x;], and py [y,]. 
From Equation 6.7.1, we have that 


3 2 
ký X (ai + 2y;) = 1, (6.7.5) 
or 
k[(1 + 2) + (1+ 4) + (2+ 2) + (244) 4+ (84 2)+ (34+4)] =1. (6.7.6) 


Therefore, k = 1/30. 
Turning to px[x;] and py [ys], 


2 
SEL aj + 2y;) = k(x; +2) + k(x; +4) = k(2z; +6) = (x; +3)/15, (6.7.7) 
where x; = 1,2,3, and 


3 
py [yy] SD. ta) = k(1 + 2yj) + k(2 + 2yj) + k(3 + 2y;) = k(6 + 6yj) = (1 + y;)/5, 


(6.7.8) 


where yj = 1,2. 


e Example 6.7.2 


Consider an urn that contains 1 red ball, 2 blue balls, and 2 green balls. Let (X,Y) 
be a bivariate random variable where X and Y denote the number of red and blue balls, 
respectively, chosen from the urn. There are 18 possible ways that three balls can be drawn 
from the urn: rbb, rbg, rgb, rgg, brb, brg, bbr, bbg, bgr, bgb, bgg, grb, grg, gbr, gbb, gbg, ggr, 
and ggb. 

The range of X and Y in the present problem is Rxy={(0, 1), (0, 2), (1,0), (1, 1), (1, 2)}. 
The joint probability mass function of (X,Y) is given by pxy[z;, yj] = P(X = i, Y = j), 
where x; = 0,1 and y; = 0,1,2. From our list of possible drawings, we find that pxy [0,0] = 
0, pxy[0, 1] = 1/6, pxy [0,2] = 1/6, pxy [1,0] = 1/6, pxy[1, 1] = 1/3, and pxy[1, 2] = 1/6. 
Note that all of these probabilities sum to one. 

Given these probabilities, the marginal probabilities are px [0] = 1/3, px[1] = 2/3, 
py [0] = 1/3, py [1] = 1/2, and py [2] = 1/3. Because pxy [0,0] 4 px [0]py [0], X and Y are 
not independent variables. 


e Example 6.7.3 


Consider a community where 50% of the families have a pet. Of these families, 60% 
have one pet, 30% have 2 pets, and 10% have 3 pets. Furthermore, each pet is equally 
likely (and independently) to be a male or female. If a family is chosen at random from 
the community, then we want to compute the joint probability that his family has M male 
pets and F female pets. 

These probabilities are as follows: 


P{F =0,M = 0} = P{no pets} = 0.5, (6.7.9) 
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P{F =1,M = 0} = P{1 female and total of 1 pet} (6.7.10) 
= P{1 pet}P{1 female|1 pet} (6.7.11) 
1 
= (0.5)(0.6) x = 0.15, (6.7.12) 
P{F = 2, M = 0} = P{2 females and total of 2 pets} (6.7.13) 
= P{2 pets} P{2 females|2 pets} (6.7.14) 
1\2 
= (0.5)(0.3) x G) = 0.0375, (6.7.15) 
and 
P{F = 3, M = 0} = P{3 females and total of 3 pets} (6.7.16) 
= P{3 pets} P{3 females|3 pets} (6.7.17) 
1\3 
= (0.5)(0.1) x (5) = 0.00625. (6.7.18) 


The remaining probabilities can be obtained in a similar manner. 


| Continuous joint distribution 


Let us now turn to the case when we have two continuous random variables. In analog 
with the definition given in Section 6.4, we define the two-dimensional probability density 


pxy (x,y) by 


P(x < X < xz+dzr,y<Y < y+ dy) =pxy(a,y) dz dy. (6.7.19) 


Here, the comma in the probability parentheses means “and also.” 
Repeating the same analysis as in Section 6.4, we find that pxy (x,y) must be a single- 
valued function with pxy (x,y) > 0, and 


/ J pxy (x,y) dz dy = 1. (6.7.20) 


The joint distribution function of X and Y is 
z py 
Fxy(z,y) = P(X < z,Y < y) = i j. pxy (£, n) d£ dn. (6.7.21) 


Therefore, 
b d 
Paa<X<be<¥<d= | f parn ddn (6.7.22) 


The marginal probability density functions are defined by 


px(z) =f pxy(x,y)dy, and r= f pxy (x,y) dx. (6.7.23) 


—oo 
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An important distinction exists upon whether the random variables are independent or not. 
Two variables X and Y are independent if and only if 


pxy (x,y) = px (x)py (y), (6.7.24) 
and conversely. 
e Example 6.7.4 
The joint probability density function of bivariate random variables (X,Y) is 


_ J kay, 0<y<a<il, 
pxy(#,¥) = { 0, otherwise, (E20) 
where k is a constant. (a) Find the value of k. (b) Are X and Y independent? 

The range Rxy for this problem is a right triangle with its sides given by + = 1, y = 0, 
and y = x. From Equation 6.7.20, 


lee) oo 1 x 1 2 |e 
/ T pxy (x,y) dx dy = ef x / yd dx = f a} de (6.7.26) 
—oco J — 00 0 0 0 2 0 
k f! 3 k aji k 


Therefore, k = 8. 
To check for independence we must first compute px (x) and py (y). From Equation 
6.7.23 and holding x constant, 


px(z)= sx | y dy = 42°, 0<x< I; (6.7.28) 
0 


px(x) = 0 otherwise. From Equation 6.7.23 and holding y constant, 


1 


py (y) = sy f xdg = 4y(1 — y’), 0<y<1. (6.7.29) 
y 


Because pyy (x,y) # px (x)py(y), X and Y are not independent. 


e Example 6.7.5: Buffon’s needle problem 


A classic application of joint probability distributions is the solution of Buffon’s needle 
problem:® Consider an infinite plane with an infinite series of parallel lines spaced a unit 
distance apart. A needle of length L < 1 is thrown upward and we want to compute the 
probability that the stick will land so that it intersects one of these lines. See Figure 6.7.1. 

There are two random variables that determine the needle’s orientation: X, the distance 
from the lower end O of the needle to the nearest line above and ©, the angle from the 
vertical to the needle. Of course, we assume that the position where the needle lands is 
random; otherwise, it would not be a probability problem. 


8 First posed in 1733, its solution is given on pages 100-104 of Buffon, G., 1777: Essai d’arithmétique 
morale. Histoire naturelle, générale et particulière, Supplément, 4, 46-123. 
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(a) intersection (b) no intersection 


Figure 6.7.1: Schematic of Buffon’s needle problem showing the random variables X and ©. 


Let us define X first. Its possible values lie between 0 and 1. Second, X is uniformly 
distributed on (0,1) with the probability density 


O0<a<l, 
0, otherwise. (6.7.30) 
Turning to O, its value lies between —7/2 to 7/2 and is uniformly distributed between these 
values. Therefore, the probability density is 


_ J l/r, —1/2<0< 7/2, 
Pel!) = { 0, otherwise. (6:731) 
The probability p that we seek is 
p = P{needle intersects line} = P{X < Lcos(©)}. (6.7.32) 


Because X and © are independent, their joint density equals the product of the densities 
for X and ©: pxyo(z,0) = px (x)po (0). 

The final challenge is to use pxe(x, 0) to compute p. In Section 6.2 we gave a geometric 
definition of probability. The area of the sample space is 7 because it consists of a rectangle 
in (X,0) space with 0 < x < 1 and —2/2 < 0 < a/2. The values of X and © that 
lead to the intersection with a parallel line is 0 < x < Lcos(@) where —7/2 < 0 < 7/2. 
Consequently, from Equation 6.2.5, 


m/2 L cos(0) m/2 L cos(0) 
p= | f pxe(x, 0) dx dé =f | px (x)pe (0) dx dé (6.7.33) 


nm /2 m/2/0 
T/2 L cos(0) 1 OL 

-1f i dz d0 = e L cos(0) d0 = F (6.7.34) 
T/2 —r/2 


Consequently, given L, we can perform the tossing either physically or numerically, measure 
p, and compute the value of 7. 
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| Convolution 


It is often important to calculate the distribution of X + Y from the distribution of 
X and Y when X and Y are independent. We shall derive the relationship for continuous 
random variables and then state the result for X and Y discrete. 

Let X have a probability density function px(x) and Y has the probability density 
py(y). Then the cumulative distribution function of X + Y is 


CC en ee E ‘i ; PXOU) de dy (6.7.35) 
=f [ rxtoronaedy = [~ | f oxla) de pr(u)dy (6.7.36) 
= f Fxla-y)pr(y) de. (6.7.37) 

Therefore, 
pirlo = |S] Fx(a~sdprtw)dy] = f" px(a—vpr(u)ay. (6.7.38) 


In the case when X and Y are discrete, 


Co 


px+ylax] = X` px{zilpy[ax — ai). (6.7.39) 


4i=—- CO 
| Covariance 


In Section 6.5 we introduced the concept of variance of a random variable X. There 
we showed that this quantity measures the dispersion, or spread, of the distribution of X 
about its expectation. What about the case of two jointly distributed random numbers? 

Our first attempt might be to look at Var(X) and Var(Y). But this would simply 
display the dispersions of X and Y independently rather than jointly. Indeed, Var(X) 
would give the spread along the x-direction while Var(Y) would measure the dispersion 
along the y-direction. 

Consider now Var(aX + bY), the joint spread of X and Y along the (ax + by)-direction 
for two arbitrary real numbers a and b. Then 


Var(aX + bY) = E[(aX + bY) — E(aX + bY)|? ( 
= E|(aX + bY) — E(aX) — EY )}? (6.7.41) 
= E{a[X — E(X)] +0[¥ - E(Y)|P? ( 
= Efa?[X — E(X)? + BY — E(Y)|? + 20b[X — E(X)|[Y — E(Y)]} 
(6.7.43) 
= a° Var(X) + b?Var(Y) + 2abE{[X — E(X)|[Y — E(Y)]}. (6.7.44) 
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Thus, the joint spread or dispersion of X and Y in any arbitrary direction ax + by depends 
upon three parameters: Var(X), Var(Y), and E{[X — E(X)|[Y — E(Y)]}. Because Var(X) 
and Var(Y’) give the dispersion of X and Y separately, it is the quantity E{[X — E(X)|[Y — 
E(Y)]} that measures the joint spread of X and Y. This last quantity, 


Cov(X,Y) = E{[X — E(X)|[Y — EYJ}, (6.7.45) 


is called the covariance and is usually denoted by Cov(X,Y) because it determines how X 
and Y covary jointly. It only makes sense when we have two different random variables 
because in the case of a single random variable, Cov(X, X) = 0% = Var(X). Furthermore, 
Cov(X, Y) < cxoy. In summary, 


Var(aX + bY) = a? Var(X) + b° Var(Y) + 2abCov(X,Y). (6.7.46) 


An alternative method for computing the covariance occurs if we recall that ux = E(X) 
and py = E(Y). Then 


Cov(X, Y) = E[(X — ux)(Y — wy)] = E(XY - uxY — py X + uxuy) (6.7.47) 
= E(XY) = pe E(Y) — pe E(X) + uxuy (6.7.48) 
= E(XY) — uxHy — Hy Hx + UxHY (6.7.49) 
= E(XY) — uxuy = E(XY) - E(X)E(Y), (6.7.50) 
where 
5 Tiyj Pxy (Li, Yj], X discrete, 
E(XY) = | “SSEB (6.7.51) 
/ J xy pxy (x,y) dz dy, X continuous. 
Therefore, 
Cov(X, Y) = E(XY) — E(X)E(Y). (6.7.52) 
It is left as a homework assignment to show that 
Cov(aX + b,cY + d) = acCov(X,Y). (6.7.53) 


In general, Cov(X,Y) can be positive, negative, or zero. For it to be positive, X 
and Y decrease together or increase together. For a negative value, X would increase 
while Y decreases, or vice versa. If Cov(X,Y) > 0, X and Y are positively correlated. If 
Cov(X,Y) <0, X and Y are negatively correlated. Finally, if Cov(X,Y) = 0, X and Y are 
uncorrelated. 


e Example 6.7.6 


The following table gives a discrete joint density function: 
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Ti 
pxy [£i yj] 0 1 2 py ly] 
3 9 3 15 
0 28 28 28 28 
3 3 3 
yy 1 FE 14 0 7 
1 1 
2 38 0 0 38 
5 15 3 
PX [x4] 14 28 28 


Because 


2 2 
3 
E(XY) = XOY riy pxy lei yy] S 
i=0 j=0 


2 

3 1 

ux = E(X) = $ eipxlei = 3, and py = E(Y) =>} ypris] = z 
= 


then 


3 9 


3 1 
Cov(X, Y) = E(XY) — E(X)E(Y) = L a 
Therefore, X and Y are negatively correlated. 


e Example 6.7.7 


The random variables X and Y have the joint probability density function 


one x+y, 0<a2<10<y<1l, 
PAYEE 0, otherwise. 


Let us compute the covariance. 
First, we must compute px (x) and py (y). We find that 


1 1 
px(a) = f pxy(au)ay = | (c+y)dy=a2+ 5 
0 0 
for 0 <a < 1, and 


1 1 
pew) = f prv(ey)de = f (@4y)de y+) 
0 0 


forO<y<l. 
Because 


Bav- f f eroaa f (I 


1 2 2 2\1 
Yy y y y 
— — 2 | dy = = iA 
[G-5)w 6+ 6 


0 3’ 


56° 


(6.7.54) 


(6.7.55) 


(6.7.56) 


(6.7.57) 


(6.7.58) 


(6.7.59) 


(6.7.60) 


(6.7.61) 


Probability 341 


px = B(x) = f vpx(a)de = f (x? +5) d= 5, (6.7.62) 

py = E(Y) -f y py (y) dy -f (v? + =) dy = (6.7.63) 
then 

CHO BE Ea a (6.7.64) 


Therefore, X and Y are negatively correlated. 


| Correlation 


Although the covariance tells us how X and Y vary jointly, it depends upon the same 
units in which X and Y are measured. It is often better if we free ourselves of this nuisance, 
and we now introduce the concept of correlation. 

Let X and Y be two random variables with 0 < oł < co and 0 < oł < oo. The 
correlation coefficient p(X, Y ) between X and Y is given by 


p(X, Y) = Cov Ž — a -5 cane ees (6.7.65) 


It is noteworthy that |p(X,Y)| < 1. 


| Random Vectors 


It is often useful to express our two random variables X and Y as a two-dimensional 
random vector V = (X Y)?. Then, the covariance can be written as a 2 x 2 covariance 
matrix, given by 


cov(X,X) cov(X,Y) 
cov(Y,X) cov(Y,Y) J` 

These considerations can be generalized into the n-dimensional random vector consisting of 

n random variables that are all associated with the same events. 


e Example 6.7.7 


Using MATLAB, let us create two random variables by invoking X = randn(N,1) and Y 
= randn(N,2), where N is the sample size. IfN = 10, we would find that using the MATLAB 
command cov(X,Y) would yield 


>> ans = 


3.1325 0.9748 

0.9748 1.4862 
(If you do this experiment, you will also obtain a symmetric matrix but with different 
elements.) On the other hand, if N = 1000, we find that cov(X,Y) equals 


>> ans = 
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Figure 6.7.2: Scatter plot of points (X;,Y;) given by the random vector V in Example 6.7.7 when N = 
1000. 


0.9793 —0.0100 
—0.0100 0.9927 ` 


The interpretation of the covariance matrix is as follows: The variance (or spread) 
of data given by X and Y is (essentially) unity. The correlation between X and Y is 
(essentially) zero. These results are confirmed in Figure 6.7.2 where we have plotted X and 
Y as the data points (X;,Y;) when N = 1000. We can see the symmetric distribution of 
data points. 


Problems 
1. A search committee of 5 is selected from a science department that has 7 mathematics 
professors, 8 physics professors, and 5 chemistry professors. If X and Y denote the number 
of mathematics and physics professors, respectively, that are selected, calculate the joint 
probability function. 
2. In an experiment of rolling a fair die twice, let Z denote a random variable that equals 
the sum of the results. What is pz[z;]? Hint: Let X denote the result from the first toss 
and Y denote the result from the second toss. What you must find is Z = X +Y. 
3. Show that Cov(aX + b,cY + d) = acCov(X,Y). 


Project: Convolution 


Consider two independent, uniformly distributed random variables (X,Y) that are summed 
to give Z = X +Y with 


oe 1, 0<a<il, 

pa 0, otherwise, 
and 

_f4, 0<y<l, 

Py (y) = { 0, otherwise. 
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0.5 


Estimated plz) 


85 0 0.5 1 1.5 2 2.5 
Zz 


Convolution Project 


Show that 
Z 0<z<1, 
pote)= | 2s Leg<y, 
0, otherwise. 


Then confirm your results using MATLAB’s intrinsic function rand to generate {z;} and {y,} 
and computing pz(z). You may want to review Example 6.5.1 in my Advanced Engineering 
Mathematics with MATLAB to see how to compute a convolution analytically. 


Further Readings 


Beckmann, P., 1967: Probability in Communication Engineering. Harcourt, Brace & World, 
511 pp. A presentation of probability as it applies to problems in communication engineer- 
ing. 


Ghahramani, S., 2000: Fundamentals of Probability. Prentice Hall, 511 pp. Nice introduc- 
tory text on probability with a wealth of examples. 


Hsu, H., 1997: Probability, Random Variables, & Random Processes. McGraw-Hill, 306 pp. 
Summary of results plus many worked problems. 


Kay, S. M., 2006: Intuitive Probability and Random Processes Using MATLAB. Springer, 
833 pp. A well-paced book designed for the electrical engineering crowd. 


Ross, S. M., 2007: Introduction to Probability Models. Academic Press, 782 pp. An intro- 
ductory undergraduate book in applied probability and stochastic processes. 


Tuckwell, H. C., 1995: Elementary Applications of Probability Theory. Chapman & Hall, 
292 pp. This book presents applications using probability theory, primarily from biology. 


Taylor & Francis 
Taylor & Francis Group 


http://taylorandfrancis.com 
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Chapter 7 


Random Processes 


In the previous chapter we introduced the concept of a random variable X. There X 
assumed various values of z according to a probability mass function py [k] or probability 
density function px (x). In this chapter we generalize the random variable so that it is also 
a function of time t. As before, the values of x assumed by the random variable X(t) at a 
certain time is still unknown beforehand and unpredictable. 

Our random, time-varying variable X(t; €) is often used to describe a stochastic or 
random process. In that case, X(t) is the state of the process at time t. The process can 
be either discrete or continuous in t. 

A random process is not one function but a collection or family of functions, called 
sample functions, with some probability assigned to each. When we perform an experiment, 
we observe only one of these functions that is called a realization or sample path of the 
process. To observe more than a single function, we must repeat the experiment. 

The state space of a random process is the set of all possible values that the random 
variable X(t) can assume. 

We can view random processes from many perspectives. First, it is a random function of 
time. This perspective is useful when we wish to relate an evolutionary physical phenomenon 
to its probabilistic model. Second, we can focus on its aspect as a random variable. This is 
useful in developing mathematical methods and tools to analyze random processes. 

Another method for characterizing a random process examines its behavior as t and € 
vary or are kept constant. For example, if we allow t and € to vary, we obtain a family or 
ensemble of X(t). If we allow t to vary while € is fixed, then X(t) is simply a function of 
time and gives a sample function or realization for this particular random process. On the 
other hand, if we fix t and allow € to vary, X(t) is a random variable equal to the state of 
the random process at time t. Finally, if we fix both t and £, then X(t) is a number. 
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0.5} 


-0.5; 


0 20 40 60 80 100 
Figure 7.0.1: A realization of the random telegraph signal. 
e Example 7.0.1 
Consider a random process X(t) = A, where A is uniformly distributed in the interval 


[0,1]. A plot of sample functions of X(t) (a plot of X(t) as a function of t) consists of 
horizontal straight lines that would cross the ordinate somewhere between 0 and 1. 


e Example 7.0.2 


Consider the coin tossing experiment where the outcomes are either heads H or tails 
T. We can introduce the random process defined by 


X(t; H) = sin(t), and X(t; T) =cos(t). (7.0.1) 


Note that the sample functions here are continuous functions of time. 


e Example 7.0.3: Random telegraph signal 


Consider a signal that switches between —a and +a at random times. Suppose the 
process starts (at time t = 0) in the —a state. It then remains in that state for a time 
interval Tı at which point it switches to the state X(t) = a. The process remains in that 
state until ¢ = Tz, then switches back to X(t) = —a. The switching time is given by a 
Poisson process, a random process that we discuss in Section 7.6. Figure 7.0.1 illustrates 
the random telegraph signal. 


Of all the possible random processes, a few are so useful in engineering and the physical 
sciences that they warrant special names. Some of them are: 


e Bernoulli process 


Imagine an electronics firm that produces electronic devices that either work (a success 
denoted by “S” ) or do not work (a failure or denoted “F”). We can model the production 
line as a series of independent, repeated events where p denotes the probability of producing 
a working device and q = 1 — p is the probability of producing a faulty device. Thus, the 
production line can be modeled as a random process, called a Bernoulli process, which has 
discrete states and parameter space. 
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Figure 7.0.2: Three realization or sample functions of a Bernoulli random process with p = 0.4. The 
realization starts at k = 0 and continues forever. The dashed box highlights the values of the random 
variable X [21, s]. 


If we denote each discrete trial by the integer k, a Bernoulli process generates successive 
outcomes at times k = 0,1,2,.... Mathematically we can express this discrete random 
process by X[k, s] where k denotes the time and s denotes the number of the realization 
or sample function. Furthermore, this random process maps the original experimental 
sample space {(F, F, S,...), (9, F, F,...), (F, F,F,...),...} to the numerical sample space 
{(0,0,1,...), (1,0,0,...),(0,0,0,...),...}. Unlike the Bernoulli trial that we examined in 
the previous chapter, each simple event now becomes an infinite sequence of S’s and F’s. 

Figure 7.0.2 illustrates three realizations or sample functions for a Bernoulli random 
variable when p = 0.4. In each realizations s = 1,2,..., the abscissa denotes time where 
each successive trial occurs at times k = 0,1,2,.... When we fix the value of k, the quantity 
X[k, s] is a random variable with a probability mass function of a Bernoulli random variable. 


e Markov process 


Communication systems transmit either the digits 0 or 1. Each transmitted digit often 
must pass through several stages. At each stage there is a chance that the digit that enters 
one stage will be changed by the time when it leaves. 

A Markov process is a stochastic process that describes the probability that the digit 
will or will not be changed. It does this by computing the conditional distribution of any 
future state Xn+1 by considering only the past states Xo, X1,...,Xn—1 and the present 
state Xn. In Section 7.4 we examine the simplest possible discrete Markov process, a 
Markov chain, when only the present and previous stages are involved. An example is the 
probabilistic description of birth and death, which is given in Section 7.5. 


e Poisson process 


The prediction of the total number of “events” that occur by time t is important to 
such diverse fields as telecommunications and banking. The most popular of these counting 
processes is the Poisson process. It occurs when: 
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1. the events occur “rarely,” 
2. the events occur in nonoverlapping intervals of time that are independent of each other, 
3. the events occur at a constant rate À. 

In Section 7.6 we explore this random process. 


e Wiener process 


A Wiener process W; is a random process that is continuous in time and possesses the 

following three properties: 

1. Wo = 0, 

2. W; is almost surely continuous, and 

3. W; has independent increments with a distribution W,—W, ~ N(0,t—s) forO<s <t. 
As a result of these properties, we have that 

1. the expectation is zero, E(W;) = 0, 

2. the variance is E(W?) — E?(W;) = t, and 

3. the covariance is cov(W,, W+) = min(s, t). 

Norbert Wiener (1894-1964) developed this process to rigorously describe the physical 
phenomena of Brownian motion—the apparent random motion of particles suspended in 
a fluid. In a Wiener process the distances traveled in Brownian motion are distributed 
according to a Gaussian distribution and the path is continuous but consists entirely of 
sharp corners. 


Project: Gambler’s Ruin Problem 


Pete and John decide to play a coin-tossing game. Pete agrees to pay John 10 cents 
whenever the coin yields a “head” and John agrees to pay Pete 10 cents whenever it 
is a “tail.” Let S» denote the amount that John earns in n tosses of a coin. This 
game is a stochastic process with discrete time (number of tosses). The state space is 
{0, +10, +20,---} cents. A realization occurs each time that they play a new game. 


Step 1: Create a MATLAB code to compute a realization of S. Plot several realizations 
(sample functions) of this random process. See Figure 7.0.3. 


Step 2: Suppose Pete has 10 dimes. Therefore, there is a chance he will run out of dimes at 
some n = N. Modify your MATLAB code to construct a probability density function that 
gives the probability Pete will run out of money at time n = N. See Figure 7.0.3. 


This problem is often formulated in terms of a gambler versus casino and called the 
gambler’s ruin problem: A gambler enters a casino with $n in cash and starts playing a 
game where he wins with probability p and loses with probability 1 — q. The gambler plays 
the game repeatedly, betting $1 in each round. He leaves the casino if his total fortune 
reaches $N or he runs out of money. 

The gambler’s ruin problem is also particularly popular because it a simple exam- 
ple of a important stochastic process called a martingale. In discrete time the martin- 
gale requires that the sequence X1, X2, X3,... satisfies two conditions: E(|X,|) < co and 
E(Xn41|X1, Xo,...,Xn) = Xn for any time n, where E(-) denotes the expectation opera- 
tor. If Xn is an observation, then we have a martingale if the conditional expected value 
of the next observation, given all the past observations, equals the most recent observation. 
To see that the gambler’s run problem is a martingale, we compute 


1 
E(Xngiltn) = 5(Xn +1) + (Xn -1)= Xn, 


NI = 
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Figure 7.0.3: (a) Top frame: John’s gains or losses as the result of the three different coin tossing games. 
(b) The probability density function for John’s winning 10 dimes as a function of the number of tosses that 
are necessary to win 10 dimes. 


where we denote the gambler’s bankroll by Xn. 


7.1 FUNDAMENTAL CONCEPTS 


In Section 6.5 we introduced the concepts of mean (or expectation) and variance as 
they apply to discrete and continuous random variables. These parameters provide useful 
characterizations of a probability mass function or probability density function. Similar 
considerations hold in the case of random processes and we introduce them here. 


| Mean and variance 


We define the mean of the random process X(t) as the expected value of the process— 
that is, the expected value of the random variable defined by X(t) for a fixed instant of 
time. Note that when we take the expectation, we hold the time as a nonrandom parameter 
and average only over the random quantities. We denote this mean of the random process 
by x(t), since, in general, it may depend on time. The definition of the mean is just the 
expectation of X(t): 


px()= EX] = | epxay(tse) de. (7.1.1) 


In a similar vein, we can generalize the concept of variance so that it applies to random 
processes. Here variance also becomes a time-dependent function defined by 


o2 (t) = Var[X(t)] = E fixt = px(OP} (7.1.2) 
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e Example 7.1.1: Random linear trajectories 


Consider the random process defined by 
X(t)=A-+ Bt, (7.1.3) 


where A and B are uncorrelated random variables with means p4 and upg. Let us find the 
mean of this random process. 
From the linearity property of expectation, we have that 


ux(t) = E[X(t)] = E(A + Bt) = E(A) + E(B)t = pa + ust. (7.1.4) 


e Example 7.1.2: Random sinusoidal signal 


A random sinusoidal signal is one governed by X(t) = Acos(wot + ©), where A and 
© are independent random variables, A has a mean j14 and variance oł, and © has the 
probability density function pe(x) that is nonzero only over the interval (0, 27). 

The mean of X(t) is given by 


px (t) = E[X(t)] = E[Acos(wot + O)|] = E[A]E|cos(wot + 0)]. (7.1.5) 
We have used the property that the expectation of two independent random variables equals 


the product of the expectation of each of the random variables. Simplifying Equation 7.1.5, 


px(t) = na | ae dx. (7.1.6) 


A common assumption is that pe(x) is uniformly distributed in the interval (0, 27), namely 


1 
polz) = oe O0<a< 2Qr. (7.1.7) 


Substituting Equation 7.1.7 into Equation 7.1.6, we find that 


27 
uxt) = al cos(wot + x) dx = 0. (7.1.8) 
0 


e Example 7.1.3: Wiener random process or Brownian motion 


A Wiener (random) process is defined by 


X(t) = I U(g)dE, = t 20, (7.1.9) 


where U(t) denotes white Gaussian noise. It is often used to model Brownian motion. To 
find its mean, we have that 


E|X(t)] = elf U(E) a = [ewe dé = 0, (7.1.10) 
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because the mean of white Gaussian noise equals zero. 


| Autocorrelation function 


When a random process is examined at two time instants t = tı and t = te, we obtain 
two random variables X (tı) and X (t2). A useful relationship between these two random 
variables is found by computing their correlation as a function of time instants tı and to. 
Because it is a correlation between the values of the same process sampled at two different 
instants of time, we shall call it the autocorrelation function of the process X(t) and denote 
it by Rx(t1,t2). It is defined in the usual way for expectations by 


Rx (ti, t2) = E[X (t1)X (t2)]. (7.1.11) 


Just as in the two random variables case, we can define the covariance and correlation 
coefficient, but here the name is slightly different. We define the autocovariance function 
as 


Cx (ti, t2) = E{[X (t1) — xx (tX (ta) — xx (ta)]} (7.1.12) 


Note that the variance of the process and its average power (the names used for the average 
of [X (t) — ux (t)]? and [X(t)]’, respectively) can be directly obtained for the autocorrelation 
and the autocovariance functions, by simply using the same time instants for both tı and 
ta: 

E{[X(t)]?} = Rx (t,t), (7.1.14) 


and 
o%(t) = E{IX() — ux (©)]?} = Cx (t,t) = Rx(t,t) - u3). (7.1.15) 


Therefore, the average power, Equation 7.1.14, and the variance, Equation 7.1.15, of the 
process follows directly from the definition of the autocorrelation and autocovariance func- 
tions. 


e Example 7.1.4: Random linear trajectories 


Let us continue Example 7.1.1 and find the autocorrelation of a random linear trajectory 
given by X(t) = A+ Bt. From the definition of the autocorrelation, 


Rx (ti, te) = E[X (t1)X (t2)] = E{[A + Bti][A + Bta]} (7.1.16) 
= E(A?) + E(AB)(ti + t2) + E(B’ )tite (7.1.17) 
= (04 + HA) + waualt + te) + (oR + wR) tite, (7.1.18) 


where o% and o% are the variances of the random variables A and B. We can easily find 
the autocovariance by 


Cx (t1, t2) — Rx (ti, ta) = [bx (t1) x (t2) = ay + otita. (7.1.19) 
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e Example 7.1.5: Random sinusoidal signal 


We continue to examine the random sinusoidal signal given by X(t) = Acos(wot + O). 
The autocorrelation function is 


Rx (ti, t2) = EIX (t1)X (tz)] = E[Acos(wot + ©)Acos(wot, + ©) (7.1.20) 
E(A?)E|cos(wote = wots) + cos(wote + woti + 20)| (7.1.21) 


II 
Nile 

q 
RNS 

+ 
= 
D 


? ){coslao(l E tı)] +f : cos|wo (ta + tı) + 2x]po (x) do), (7.1.22) 


In our derivation we used (1) the property that the expectation of A? equals the sum of 
the variance and the square of the mean, and (2) the first term involving the cosine is 
not random because it is a function of only the time instants and the frequency. From 
Equation 7.1.22 we see that autocorrelation function may depend on both time instants if 
the probability density function of the phase angle is arbitrary. On the other hand, if pe (x) 
is uniformly distributed, then the last term in Equation 7.1.22 vanishes because integrating 
the cosine function over the interval of one period is zero. In this case we can write the 
autocorrelation function as a function of only the time difference. The process also becomes 
wide-sense stationary with 


Rx(r) = E[X(t)X(t + 7)] = $(0% + u3) cos(wor). (7.1.23) 


| Wide-sense stationary processes 


The mathematical analysis of a random or stochastic process would appear to be hope- 
less because of the uncertainty of its time-dependent behavior at any instant of time. To 
circumvent this difficulty we will examine only those processes that have certain statistical 
properties at any instant. A wide-sense stationary process is one of the most popular. 

A process is strictly stationary if its distribution and density functions do not depend 
on the absolute values of the time instants tı and t2, but only on the difference of the time 
instants, |tı — t2|. However, this is a very rigorous condition. If we are concerned only with 
the mean and autocorrelation function, then we can soften our definition of a stationary 
process to a limited form, and we call such processes wide-sense stationary processes. A 
wide-sense stationary process has a constant mean, and its autocorrelation function depends 
only on the time difference: 


x(t) = E[X(t)] = wx, (7.1.24) 


and 
Rx (ti, te) = E[X (t1)X (t2)] = Rx (t2 - th). (7.1.25) 


Because time does not appear in the mean, we simply write it as a constant mean value px. 
Similarly, because the autocorrelation function is a function only of the time difference, we 
can write it as a function of a single variable, the time difference 7T: 


Rx(T) = E[X(t)X(t +7). (7.1.26) 
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We can obtain similar expressions for the autocovariance function, which in this case de- 
pends only on the time difference as well: 


Cx (7) = E{[X(t) — uxl[X (t +7) — ux]} = Rx(7) — uk. (7.1.27) 
Finally, the average power and variance for a wide-sense stationary process are 
E{[X(t)]}?} = Rx(0), and oy =Cx(0) = Rx(0) — ny, (7.1.28) 


respectively. Therefore, a wide-sense stationary process has a constant average power and 
constant variance. 


Problems 
1. Find x(t) and oł (t) for the random process given by X(t) = Acos(wt), where w is 


a constant and A is a random variable with the Gaussian (or normal) probability density 
function 


2. Consider a sine-wave random process X(t) = Acos(wt + O), =r < t < r, where A and 
w are constants with A > 0. The phase function © is a random, uniform variable on the 
interval [—7,7]. Find the mean, variance and autocorrelation for this random function. Is 
this process wide-sense stationary? 


3. Consider a countably infinite sequence {X,,n = 0,1,2,3,...} of a random variable 
defined by 
ya { 1, for success in the nth trial, 
"| 0, for failure in the nth trial, 


with the probabilities P(X, = 0) = 1 — p and P(X, = 1) = p. Thus, X, is a Bernoulli 
process. For this process, E(X,,) = p and Var(X,) = p(1—p). Show that the autocorrelation 
is 


P, tı > t2, 
Rx (ti, t2) = 
x (ty ta) { r, ty A to; 
and the autocovariance is 
p(l- p), ty = te, 
ti, t2) = 
Cx( 1; 2) { 0, ty Æ t2 


Project: Computing the Autocorrelation Function 


In most instances you must compute the autocorrelation function numerically. The 
purpose of this project is to explore this computation using the random telegraph signal. 
The exact solution is given by Equation 7.2.24. You will compute the autocorrelation two 
ways: 


Step 1: Using Example 7.6.1, create MATLAB code that generates 500 realizations of the 
random telegraph signal. 
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0.2 0.4 0.6 0.8 


Figure 7.1.1: The autocorrelation function Rx (rT) for the random telegraph signal as a function of r when 
A = 2. The dashed line gives the exact solution. In frame (a) X;(ts)Xz(ts +7) has been averaged over 
500 realizations when tg = 2. In frame (b) X209(mAt)X200(mAt + T) has been averaged with M = 1200 
and At = 0.01. 


Step 2: Choosing an arbitrary time ts, compute X;,(ts)Xz(ts +7) for 0 < 0 < Tmax and 
k = 1,2,3,...,500. Then find the average value of X;,(tg)X;(ts +7). Plot Rx(r) asa 
function of 7 and include the exact answer for comparison. Does it matter how many sample 
functions you use? 


Step 3: Now introduce a number of times tm = mAt, where m = 0,1,2,..., M. Using only 
a single realization k = K of your choice, compute Xx (mAt) x Xx (mAt+7). Then find 
the average value of Xx (mAt)XxK(mAt+7) and plot this result as a function of 7. On the 
same plot, include the exact solution. Does the value of At matter? See Figure 7.1.1 


7.2 POWER SPECTRUM 


In earlier chapters we provided two alternative descriptions of signals, either in the 
time domain, which provides information on the shape of the waveform, or in the frequency 
domain, which provides information on the frequency content. Because random signals do 
not behave in any predictable fashion nor are they represented by a single function, it is 
unlikely that we can define the spectrum of a random signal by taking its Fourier transform. 
On the other hand, the autocorrelation of random signals describes in some sense whether 
the signal changes rapidly or slowly. In this section we explain and illustrate the concept 
of power spectrum of random signals. 

For a wide-sense stationary random signal X(t) with autocorrelation function Rx (rT), 
the power spectrum Sx (w) of the random signal is the Fourier transform of the autocorre- 
lation function: m 

Sx(w) = / Rx (r)e7**” dr. (7.2.1) 
— 00 
Consequently, the autocorrelation can be obtained from inverse Fourier transform of the 
power spectrum, or 


Rx(r) = = L. Sx (w) dw. (7.2.2) 
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As with any Fourier transform, it enjoys certain properties. They are: 

1. The power spectrum is real and even: Sx(—w) = Sx(w) and SX (w) = Sx(w), where 
S% (w) denotes the complex conjugate value of Sx (w). 

2. The power spectrum is nonnegative: S'x(w) > 0. 


3. The average power of the random signal is equal to the integral of the power spectrum: 
1 co 
E{[X(t)|?} = Rx (0) = F Sx (w) dw. (7.2.3) 
0 


4. If the random signal has nonzero mean ux, then its power spectrum contains an impulse 
at zero frequency of magnitude 2r u% . 


5. The Fourier transform of the autocovariance function of the random process is itself 
also a power spectrum and usually does not contain an impulse component in zero 
frequency. 


Consider the following examples of the power spectrum: 
e Example 7.2.1: Random sinusoidal signal 
The sinusoidal signal is defined by 
X(t) = Acos(wot + O), (7.2.4) 


where the phase is uniformly distributed in the interval [0,27]. If the amplitude A has a 
mean of zero and a variance of a”, then the autocorrelation function is 


Rx(T) = $07 cos(woT) = Rx (0) cos(wor). (7.2.5) 


The power spectrum of this signal is then 
Sx(w) = / Rx (0) cos(wot)e~*”” dr = Rx (0)r [5(w — wo) + lw + wo)]. (7.2.6) 


Because this signal contains only one frequency wo, its power spectrum is just two impulses, 
one at wo and one at —wọ. Since the negative frequency appears due only to the even 
property of the power spectrum, it is clear that all power is concentrated at the frequency 
of the sinusoidal signal. While this is a very simple example, it does illustrate that the 
power spectrum indeed represents the way the power in the random signal is distributed 
among the various frequencies. We shall see later that if we also use linear systems in 
order to amplify or attenuate certain frequencies, the results mirror what we expect in the 
deterministic case. 


e Example 7.2.2: Modulated signal 


Let us now examine a sinusoidal signal modulated by another random signal that 
contains low frequencies. This random process is described by 


Y(t) = X (t) cos(wot + O), (7.2.7) 
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where the phase angle in Equation 7.2.7 is a random variable that is uniformly distributed 

in the interval [0, 27] and is independent of X(t). Then the autocorrelation function of Y(t) 

is given by 

Ry(r) = E[Y(t)Y (t+ T)| = E{X (t) cos(wot + O)X (t + T) cos[wo(t + T) + O]} (7.2.8) 
= E[X (t)X(t + 7)|E{cos(wot + O) cos[wo(t +7) + O]} = $Rx(rT)cos(wot). (7.2.9) 


Let us take Rx(T) = Rx(0)e~?!7!, the autocorrelation function for a random telegraph 
signal (see Equation 7.2.22). In this case, 
Ry(T) = LRx(0)e "7! cos(wot). (7.2.10) 


Turning to the power spectrum, the definition gives 


Saar / 1 Rx (r) cos(wot)e“™" dr (7.2.11) 
= if Rx (T) (e07 + eT) eM dr (7.2.12) 
= 3 [Sx w% — wo) + Sx (w + wo)]. (7.2.13) 


Thus, the resulting power spectrum is shifted to the modulating frequency wo and its 
negative value, with peak values located at both w = wo and w = —wo. 


e Example 7.2.3: White noise 


There are instances when we want to approximate random signals where the autocor- 
relation function is very narrow and very large about 7 = 0. In those cases we construct an 
idealization of the autocorrelation function by using the impulse or delta function d(T). 

In the present case when Rx (rT) = C'd(r), the power spectrum is 


Sx(w) = E C ôlr)e™™T dr = C. (7.2.14) 


Thus, the power spectrum here is a flat spectrum whose value is equal to C. Because the 
power spectrum is flat for all frequencies, it is often called “white noise” since it contains 
all frequencies with equal weight. 

An alternative derivation involves the random telegraph that we introduced in Example 
7.0.3. As the switching rate becomes large and the rate À approaches infinity, its amplitude 
increases as vÀ. Because Ry(0) increases linearly with A, the autocorrelation function 
becomes 

Rx (rT) = Crexp(—2AI7]). (7.2.15) 


The resulting power spectrum equals 


4 2 
A E ee os 2 


— =C. .2.16 
RAR AAO vee) 


The power spectrum is again flat for all frequencies. 
The autocorrelation for white noise is an idealization because it has infinite average 
power. Obviously no real signal has infinite power since in practice the power spectrum 
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decays eventually. Nevertheless, white noise is still quite useful because the decay usu- 
ally occurs at such high frequencies that we can tolerate the errors of introducing a flat 
spectrum. 


e Example 7.2.4: Random telegraph signal 


In Example 7.0.3 we introduced the random telegraph signal: X(t) equals either +h or 
—h, changing its value from one to the other in Poisson-distributed moments of time. The 
probability of n changes in a time interval 7 is 


P.(n) = QU" ar (7.2.17) 


n! : 


where A denotes the average frequency of changes. 

To compute the power spectrum, we must first compute the correlation function via 
the product X(t)X(t +7). This product equals h? or —h?, depending on whether X(t) = 
X(t +7) or X(t) = —X(t+7), respectively. These latter relationships depend on the 
number of changes during the time interval. Now, 


n 


( 


PIX (t) = X(t +7)] = P, (n even) = e^ T = =e ** cosh(Ar), (7.2.18) 


and 


P[X(t) = -X(t + 7)] =P,(n odd) =e" $. =e sinh(Ar). (7.2.19) 


= (2n+1)! 
Therefore, 
E[X(t)X(t +7)] = h? P, (n even) — h? P, (n odd) (7.2.20) 
= h?e™™ [cosh(Ar) — sinh (àr )] (7.2.21) 
= he AI, (7.2.22) 


We have introduced the absolute value sign in Equation 7.2.24 because our derivation was 
based on t2 > tı and the absolute value sign takes care of the case t2 < ty. 
Using Problem 1, we have that 


Sx(w) = 2h? j —?AT cos(AT) d Le (7.2.23) 
x(w) = A e S(AT T= PA Ai 


Problems 
1. Show that “2 
Sx(w) = al Rx (T) cos(wr) dr. 
0 
7.3 TWO-STATE MARKOV CHAINS 


A Markov chain is a probabilistic model in which the outcomes of successive trials 
depend only on its immediate predecessors. The mathematical description of a Markov 
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chain involves the concepts of states and state transition. If X,, = i, then we have a process 
with state 7 and time n. Given a process in state i, there is a fixed probability P;; that state 
i will transition into state j. In this section we focus on the situation of just two states. 

Imagine that you want to predict the chance of rainfall tomorrow.! From close obser- 
vation you note that the chance of rain tomorrow depends only on whether it is raining 
today and not on past weather conditions. From your observations you find that if it rains 
today, then it will rain tomorrow with probability a, and if it does not rain today, then 
the chance it will rain tomorrow is 8. Assuming that these probabilities of changes are 
stationary (unchanging), you would like to answer the following questions: 


1. Given that it is raining (or not raining), what are the chances of it raining in eight 
days? 


2. Suppose the day is rainy (or dry). How long will the current weather remain before it 
changes for the first time? 


3. Suppose it begins to rain during the week. How long does it take before it stops? 


If the weather observation takes place at noon, we have a discrete parameter process; 
the two possible states of the process are rain and no rain. Let these be denoted by 0 for no 
rain and 1 for rain. The four possible transitions are (0 —> 0), (0 > 1), (1 > 0), and (1 > 1). 
Let X, be the state of the process at the nth time point. We have X, = 0,1. Clearly, 
{X,,n =0,1,2,...} is a two-state Markov chain. Therefore, questions about precipitation 
can be answered if all the properties of the two-state Markov chains are known. Let 


Pare =a. BS ee. (7.3.1) 


ve m) denotes the probability that the state of the process at the nth time point is 7 given 
that it was at state i at the mth time point. Furthermore, if this probability is larger for 
i = j than when i Æ j, the system prefers to stay or persist in whatever state it is. When 
n =m + 1, we have that 


ES POG ia =a (7.3.2) 


This is known as the one-step transition probability, given that the process is at į at time 
m. 

There are two possibilities: either pr ce) ) depends on m or poe mD is independent 
of m, where m is the initial value of the time parameter. Our precipitation model is an 
example of a second type of process in which the one-step transition probabilities do not 
change with time. Such processes are known as time homogeneous. Presently we shall 
restrict ourselves only to these processes. Consequently, without loss of generality we can 


use the following notation for the probabilities: 


and 
po = P(Xmin = j| Xm = 1) for all m. (7.3.4) 


1 See, for example, Gabriel, K. R., and J. Neumann, 1962: A Markov chain model for daily rainfall 
occurrence at Tel Aviv. Quart. J. R. Met. Soc., 88, 90-95. 
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| Chapman-Kolmogorov equation 


The Chapman?-Kolmogorov? equations provide a mechanism for computing the tran- 


sition probabilities after n steps. The n-step transition probabilities Pe denote the prob- 
ability that a process in state i will be in state j after n transitions, or 


PP = PiXnie =5|Xe=i], 020, 4,920. (7.3.5) 
Therefore, po) = P;j. The Chapman-Kolmogorov equations give a method for computing 
these n-step transition probabilities via 


n+m n m 
PS = 2, Py E 2 n,m > 0, (7.3.6) 
k=0 


for all i and j. Here pe ee represents the probability that the ith starting process will 
go to state j in n+™m transitions via a path that takes it into state k at the nth transition. 
Equation 7.3.6 follows from 


CO 


POY 2 Pl Xan 9X0) = YP an Si Aw Sh oS (7.3.7) 
k=0 
=y P = j|\Xn =k, Xo a EER (7.3.8) 
k=0 k=0 
| Transmission probability matrix 
Returning to the task at hand, we have that 
PË = POHD = P. P = P?, (7.3.9) 
and by induction 

P® = pe) = pe-1). P = pr, (7.3.10) 


where P“™) denotes the transition matrix after n steps. 

From our derivation, we see the following: (1) The one-step transition probability 
matrix completely defines the time-homogeneous two-state Markov chain. (2) All transition 
probability matrices show the important property that the elements in any of their rows add 
up to one. This follows from the fact that the elements of a row represent the probabilities 
of mutually exclusive and exhaustive events on a sample space. 


2 Chapman, S., 1928: On the Brownian displacements and thermal diffusion of grains suspended via 
non-uniform fluid. Proc. R. Soc. London, Ser. A, 119, 34-54. 


3 Kolmogorov, A. N., 1931: Uber die analytischen Methoden in der Wahrscheinlichkeitsrechnung. Math. 
Ann., 104, 415-458. 
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Table 7.3.1: The Probability of Rain on the nth Day. 


n Poo Pro Poi Pu 

1 0.7000 0.2000 0.3000 0.8000 
2 0.5500 0.3000 0.4500 0.7000 
3 0.4750 0.3500 0.5250 0.6500 
4 0.4375 0.3750 0.5625 0.6250 
5 0.4187 0.3875 0.5813 0.6125 
6 0.4094 0.3938 0.5906 0.6063 
7 0.4047 0.3969 0.5953 0.6031 
8 0.4023 0.3984 0.5977 0.6016 
9 0.4012 0.3992 0.5988 0.6008 
10 0.4006 0.3996 0.5994 0.6004 
ee) 0.4000 0.4000 0.6000 0.6000 


For two-state Markov processes, this means that 


PY POS sand: BPP St (7.3.11) 
Furthermore, with the one-step transmission probability matrix: 
pas! oy O<ab<1, |l-a-b|<1, (7.3.12) 


then the n-step transmission probability matrix is 


n n b (1—a—b)” a (1—a—b)” 
(8, Fat) = ee a+b a+b q a+b ) . (7.3.13) 


ARRA Aea T a 


a 


This follows from the Chapman-Kolmogorov equation that 


P® =1-a, (7.3.14) 

and 
PE = (1-a) PQY +P Y,  n>1, (7.3.15) 
=b+ (1-a b) PP, (7.3.16) 


since pe ha 1- PO ), Solving these equations recursively for n = 1, 2,3,... and simplifying, 


we obtain Equation 7.3.13 as long as both a and b do not equal zero. 
e Example 7.3.1 


Consider a precipitation model where the chance for rain depends only on whether it 
rained yesterday. If we denote the occurrence of rain by state 0 and state 1 denotes no rain, 
then observations might give you a transition probability that looks like: 


0.7 0.3 
P= (ns Loe (7.3.17) 


Given that the atmosphere starts today with any one of these states, the probability of 
finding that it is raining on the nth day is given by P”. Table 7.3.1 illustrates the results 
as a function of n. Thus, regardless of whether it rains today or not, in ten days the chance 
for rain is 0.4 while the chance for no rain is 0.6. 
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| Limiting behavior 


As Table 7.3.1 suggests, as our Markov chain evolves, it reaches some steady state. Let 
us explore this limit of n — oo because it often provides a simple and insightful represen- 
tation of a Markov process. 

For large values of n it is possible to show that the limiting probability distribution of 
states is independent of the initial value. In particular, for |1 — a — b| < 1, we have that 


a 


b _a_ 

lim PP!) = ( ob atb), 7.3.18 

b 
a+b a+b 


This follows from limpo (1 — a — b)” + 0 since |1 — a — b| < 1. From Equation 7.3.13 the 
second term in each of the elements of the matrix tends to zero as n > co. 
Let us denote these limiting probabilities by 7; = limn—oo P”, Then, from Equation 


ij 
7.3.18, 
b d a 
—— =r nd Tol = T11 = — = 
PEE 0; 01 TEE 


and these limiting distributions are independent of the initial state. 


| Number of visits to a certain state 


When a random process visits several states, we would like to know the number of 


visits to a certain state. Let NS denote the number of visits the two-state Markov chain 


{Xn} makes to state j, starting initially at state i, in n time periods. If TA denotes the 
expected number of visits that the process makes to state j in n steps after it originally 
started at state i, and the transition probability matrix P of the two-state Markov chain is 


p=("5° <3 (7.3.20) 


Too = 710 = Ti, (7.3.19) 


with |1 — a — b| < 1, then 


(n) ne + ee ae na aise bi G=] 
leis Il = ( nb _ (1—a~b)[1—(1=a-b)"]_ na y b(1-a~8)[1~(1-a—B)”] ) (7.3.21) 
a+b (a+b)? a+b (a+b)? 
To prove Equation 7.3.21, we introduce a random variable ya where 
(rn) Jl, if Xk = j and Xo = å, 
tg = T otherwise, (7.3.22) 


for i,j = 0,1. This random variable yo 


The probability distribution of Yọ” for fixed k is 


gives the time at which the process visits state j. 


(n) 
y, 0 1 
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Probability 1- P” po 
Thus, we have that 
EY] = 2 i,j =0,1; k=1,2,...,n. (7.3.23) 


Because ye equals 1 whenever the process is in state j and 0 when it is not in j, the 
number of visits to j, starting originally from 7, in n steps is 


(n) _ yQ) (2) (n) 
Na Nae PG eer as (7.3.24) 
Taking the expected values and using the property that the expectation of a sum is the sum 


of expectations, 


a tj 


wy SEINE) a a = PM. (7.3.25) 
k=1 


From Equation 7.3.13, we substitute for each po and find 


n L p(k z b a(l — a — b)? 
iio = 5 P = 5 | ae + Paar ; (7.3.26) 
k=1 k=1 
n Z pk = a a(1—a—b)* 
ENB = | L a h (7.3.27) 
k=1 k=1 
n o Š Tf b bese) 
i= yey = E z at | (7.3.28) 
k=1 k=1 
and n 
n) (k) a b(1—a—b)* 
See 7.3.29 
a 2 g 2 | +b a+b , ( ) 
k=1 k=1 
finally, noting that 
a+b a+b 
k=1 
and 
is a(l —-a-— b)” a 5 k 
2 E (1—a-—b) (7.3.31) 
A a+b a+b 
= (l1—a—b)+(1—-a—b)?+---+(1-a—b)"] (7.3.32) 
a(l1—a—b) pa 
=p eieae Aege (7.3.33) 


_ a(l—a-—b)[1-— (1-— a- b)”] 
= (a+b)[(1-G-—a-b)) ` (7.3.34) 


Here we used the property of a geometric series that 


n-1 
1 — N 

Ya, jeji (7.3.35) 

r 1-2 
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e Example 7.3.2 


Let us continue with our precipitation model that we introduced in Example 7.3.1. If 
we wish to know the expected number of days within a week that the atmosphere will be 
in a given state, we have from Equation 7.3.21 that 


= ay = 3.3953, (7.3.36) 
m 1. a= b= (b= a0) 
= ay = 2.4031, (7.3.37) 
~_ 7a _a(l—a—dfl-(1-a-d)]_ 4, 7.3.38 
wh = 2 ay 3.6047, (7.3.38) 
and ( ) ( iA 
mo Ta  1-a—b)[l—-(1-a—6)7] _ 
w= + ay: = 4.5969, (7.3.39) 


since a = 0.3 and b = 0.2. 


| Duration of stay 


In addition to computing the number of visits to a certain state, it would also be useful 
to know the fraction of the discrete time that a process stays in state j out of n when the 
process started in state 7. These fractions are: 


(n) (n) 
lim 700 — Jim #0 = To, (7.3.40) 
n>c 71 n> n 
and 
us) ne 
lim = lim = ay. (7.3.41) 
n>co n n> 1 


Thus, the limiting probabilities also give the fraction of time that the process spends in the 
two states in the long run. 

If the process is in state i (i = 0,1) at some time, let us compute the number of 
additional time periods it stays in state ¿i until it moves out of that state. We now want to 
show that this probability distribution a;, i = 0,1, is 


P(ag =n) =a(l—a)”, (7.3.42) 
and 
P(a; =n) =b(1 — b)”, (7.3.43) 
where n = 1,2,3,.... Furthermore, 
E(ao)=(1—a)/a,  E(aı) = (1 — b)/b, (7.3.44) 
and 


Var(ao) = (1 — a)/a?, Var(a1) = (1 — b)/b?, (7.3.45) 
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where the transition probability matrix P of the Markov chain {X,,} equals 


Pata A) (7.3.46) 


with |1 — a — b| < 1. Clearly a or b cannot equal zero. 

To prove this we note that at every step, the process has two choices: either to stay in 
the same state or to move to the other state. Suppose the process is in state 0 at some time. 
The probability of a sequence of outcomes of the type {00---01} is required. Because of the 


property of Markov-dependence, we therefore have the realization of a Bernoulli process with 
n consecutive outcomes of one type followed by an outcome of the other type. Therefore, 
the probability distribution of ao is geometric with (1 — a) as the probability of “failure,” 
and the distribution of a is geometric with (1— b) as the probability of failure. Thus, from 
Equation 6.6.5, we have that 


P(ap =n) = a(1 — a)”, (7.3.47) 
and 

P(a, =n) = b(1 — b)”, (7.3.48) 
where n = 0,1,2,.... The expressions for the mathematical expectation and variance of ag 


and qı easily follow from the corresponding expressions for the geometric distribution. 
e Example 7.3.3 


Let us illustrate our expectation and variance expressions for our precipitation model. 
From Equation 7.3.44 and Equation 7.3.45, we have that 


E(ao) = (1 — a)/a = 2.3333, — E(ay) = (1 — b)/b = 4, (7.3.49) 


nd 
i Var(ao) = (1 — a)/a? = 7.7777, — Var(aı) = (1 — b) /b? = 20, (7.3.50) 


since a = 0.3 and b = 0.2. 


e Example 7.3.4: Gambler’s ruin problem 


At the beginning of this chapter we introduced the gambler’s ruin problem as an ex- 
ample of a random process. Here we wish to redo that problem as a Markov chain 

Our particular version of the game is as follows: A gambler plays a game involving the 
flipping of a coin. The probability of the coin coming up heads is p while the probability of 
the coin coming up tails is q = 1— p. He enters the game with some initial amount of money 
and plays until (1) he has lost all of his money or (2) he has gained N units of money. We 
would like to describe this game at the j flip of the coin. 

Let x; denote the probability that the gambler has i units of money. At the j flip, 
these probabilities will be affected by the states x;}ı and x;_, according to 

alt) pel tqel_,, i=1,2,...,.N-1. (7.3.51) 

Equation 7.3.51 does not describe the states 1 = 0 and i = N, the absorbing states. State 
i = 0 corresponds to the gambler losing all of his money and quitting the game while state 
i = N corresponds to the gambler winning N units of money and calling it a night. Once 
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DOeGO-GOD 


Figure 7.3.1: Markov chain diagram for the gambler’s ruin problem. 


these absorbing states are attained, there is no way of going to another state: ait? = xÍ 


and ait = ah. Since these absorbing states can be eventually reached from any other 
state, the game will eventually reach a steady state. We have illustrated this Markov chain 
in Figure 7.3.1. 

The most convenient way of computing x is via matrix algebra. Using matrix notation, 


we can compute the probabilities from: 


xt! = xÍ P, (7.3.52) 
where 
100 0 0 0 0 0 0 
q 0 p00 .- 0 0 0 0 
0 q 0 0 0 0 0 0 
Bee ee pu (7.3.53) 
000 0 0 q 0 p O 
000 0 0 :-- 0 q¢ 0 p 
00000 >. 00 0 1 
and x is the row vector [vp £1 +--+ £y-1£y]. 


To illustrate the evolution of the gambler’s ruin problem, let us set N = 3, p= q = 0.5, 
and xo = [0 1 0 0]?. Then. 


xı = XoP = [0.5 0 0.5 0] (7.3.54) 

x2 = xo P?’ = [0.5 0.25 0 0.25] (7.3.55) 

x3 = xo P” = [0.625 0 0.125 0.25] (7.3.56) 

X19 = XoP!° = [0.66601562 0.00097656 0 0.33300781] (7.3.57) 

X100 = Xo P1% = [0.6660666 0.00000000 0.00000000 0.3333333]. (7.3.58) 


The interpretation of these results is straightforward. After 100 games, the probability 
that the gambler, with an initial bankroll of one unit of money, will lose all his money is 
2/3 while the chance that he will go home with 3 units of money is 1/3. There are no other 
outcomes to the game. 


Problems 
1. Given 
p= 3/4 1/4 
ONG: 1/2)” 


(a) compute P” and (b) find lim,-,.. P”. 


2. Suppose you want to model how your dog learns a new trick. Let Fido be in state 0 if 
he learns the new trick and in state 1 if he fails to learn the trick. Suppose that if he learns 
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the trick, he will retain the trick. If he has yet to learn the trick, there is a probability a 
of him learning it with each training session. (a) Write down the transition matrix. (b) 
Compute P(™ where n is the number of training sessions. (c) What is the steady-state 
solution? Interpret your result. (d) Compute the expected amount of time that Fido will 
spend in each state during n training sessions. 


7.4 BIRTH AND DEATH PROCESSES 


In the previous section we considered two-state Markov chains that undergo n steps. 
As the time interval between steps tends to zero, the Markov process becomes continuous 
in time. In this section and the next, we consider two independent examples of continuous 
Markov processes. 

We began Chapter 6 by showing that the deterministic description of birth and death 
is inadequate to explain the extinction of species. Here we will fill out the details of our 
analysis and extend them to population dynamics and chemical kinetics. Deterministic 
models lead to first-order ordinary differential equations, and this description fails when 
the system initially contains a small number of particles. 

Consider a population of organisms that multiply by the following rules: 


1. The sub-populations generated by two co-existing individuals develop completely in- 
dependently of one another; 

2. an individual existing at time t has a chance  dt+o(dt) of multiplying by binary fission 
during the following time interval of length dt; 

3. the “birth rate” A is the same for all individuals in the population at any time t; 

4. an individual existing at time t has a chance u dt + o(dt) of dying in the following time 
interval of length dt; and 

5. the “death rate” u is the same for all individuals at any time t. 


Rule 3 is usually interpreted in the sense that in each birth, just one new member is added 
to the population, but of course mathematically (and because the age structure of the 
population is being ignored) it is not possible to distinguish between this and an alternative 
interpretation in which one of the parents dies when the birth occurs and is replaced by 
two children. 

Let no be the number of individuals at the initial time t = 0 and let p,(t) denote the 
probability that the population size N(t) has the value n at the time t. Then 


dpn 
Tr = (n — 1)àpn-1 — N(À + H)pn + u(n + 1)Pn+1, n>1, (7.4.1) 
and dpo(t) 
Do(t 
= t 7.4.2 
q H(t); ( ) 
subject to the initial condition that 
_ Jd, n = no, 
Pn(0) = K EP (7.4.3) 


Equation 7.4.1 through Equation 7.4.3 constitute a system of linear ordinary equations. 
The question now turns on how to solve them most efficiently. To this end we introduce a 
probability-generating function: 


olz, t) = X. 2" pale). (7.4.4) 


n=0 
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Summing Equation 7.4.1 from n = 1 to oo after we multiplied it by z” and using Equation 
7.4.2, we obtain 


Se "ia A2 (n—1)2"pn—i(t)—(A+H) X` nz” p(t) +H X (n+ 12"pngi(t). (7.4.5) 


n=0 n=1 n=0 
Because 
nn _ 3$ 
n = .4.6 
2 ede Ok ee) 
a“ = poe = 2t, (7.4.7) 
20 — 1)z”pn-1( => kz¥+1y,(t) = 2? 2 kz”! p(t) = et, (7.4.8) 
and m es 
So (n+ 1)” Pailt => kz*—*p,(t) = S > ka*1p, (t) = 7 (7.4.9) 
4 
n=0 k=0 


Equation 7.4.5 becomes the first-order partial differential equation 


(7.4.10) 


Oz’ 
subject to the initial condition 


o(z,0) = 2". (7.4.11) 


Equation 7.4.10 is an example of a first-order partial differential equation of the general 
form 


Ou ð 
P(t, y)a- ar Q(z, y) 7S =0. (7.4.12) 


This equation has solutions‘ of the form u(x, y) = f(€) where f(-) is an arbitrary function 
that is differentiable and (x, y) = constant are solutions to 


dx dy 


P(x,y)  Q(a,y)’ C 
In the present case, 
dt dz dz dz 
L Rene aee uN a 
Integrating Equation 7.4.14, 
—(A— u)t + In[y(z)] = In(€), (7.4.15) 
or 
Elz, t) = p(T Ves (7.4.16) 


4 See Webster, A. G., 1966: Partial Differential Equations of Mathematical Physics. Dover, 446 pp. 
See Section 22. 
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where ‘ 
zap 
= : 7.4.17 
ge) = 7 (7.4.17) 
Therefore, the general solution is 
plz, t) = fye]. (7.4.18) 


Our remaining task is to find f(-). From the initial condition, Equation 7.4.11, we have 
that 


o(z,0) = fly(z)] = 2™. (7.4.19) 
Because z = [u — y(z)]/[A — y(z)], then 
f) = (=) f (7.4.20) 
Therefore, 
— y(z)e7 O- no 
o(z,t) = É = ed (7.4.21) 


Once we find ¢(z,t), we can compute the probabilities of each of the species from the 
probability generating function. For example, 


P{N(t) = 0|N(0) = no} = po(t) = (0, t). (7.4.22) 


From Equation 7.4.17 we have w(0) = u and 


no 
ui felis) 
$(0, t) Sete p >. Wee (7.4.23) 
and 
Wasa. Ke (7.4.24) 
Be > Nae i Ti: 7 
An important observation from Equation 7.4.23 and Equation 7.4.24 is that 
Jim po(t)=1, ASH, (7.4.25) 
and 
Í py” 
lim polt) = (£) TE DS (7.4.26) 
t= oo A 


This limit can be interpreted as the probability of extinction of the population in a finite 
time. Consequently, there will be “almost certain” extinction whenever A < p. These 
results, which are true whatever the initial number of individuals may be, show very clearly 
the inadequacy of the deterministic description of population dynamics. 

Finally, let us compute the mean and variance for the birth and death process. The 
expected number of individuals at time t is 


m(t) = EIN(#)] = Y n palt) = X n pal). (7.4.27) 
n=0 n=] 
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Now 
eee n— = n|(n — iy n t n t } n nEs 
de ug Oe pee i E ae 


= AX (n — 1)?pn-1 +A Xoin — 1)pn-1 — (A+ p) ba npn + u (n +1)? Pny 


n=1 n=1 n=1 n=1 


—p So (n+p (7.4.29) 


n=l 


=-—(à+ p) DD n? Pn + AX PPm +u» k’°pr + AX ipi — D k pp. (7.4.30) 
n=1 i=0 k=2 i=0 k=2 


In the first three sums in Equation 7.4.30, terms from i, k,n = 2, and onward cancel and 
leave —(A + u)pı + Ap, = —upı. Therefore, 


dm 


1=0 k=2 n=0 


If we choose the initial condition m(0) = no, the solution is 
m(t) Ses (7.4.32) 
This is the same as the deterministic result with the birth rate b replaced by \ and the death 


rate d replaced by u. Furthermore, if A = u, the mean size of the population is constant. 
The second moment of N(t) is 


M(t) = Se npn (t). (7.4.33) 
n=0 
Proceeding as before, we have that 


Co d PS Co 
= SE = Son? Aln- Ipni — A+ unpa + u(n + Vpn (7.4.34) 


n=1 n=1 


= ASC (n — 1)?pn-1 + 2A Xn — 1)?pn-1 +A Xin — 1)pn-1 — (A+ y) 5 Dn 
n=l n=1 n=l n=1 


+p Son +1)? pas — 2u Son +1) pny +u Xn +1)pn41 (7.4.35) 
n=1 n=1 n=1 
=A X TR pp + 2S 7 kpr +AS kpr- (AFB) YS npa 
k=1 k=1 k=1 n=1 
+ n> Ppi- 25° Pp + ud ipi. (7.4.36) 
1=2 J= 4=2 


The three sums, which contain either i? or k? or n° in them, cancel when i,k,n = 2 and 
onward; these three sums reduce to —upı. The sums that involve i? or k? can be written 
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in terms of M(t). Finally, the sums involving i and k can be expressed in terms of m(t). 
Therefore, Equation 7.4.36 becomes the first-order ordinary differential equation 

dM 

= 20) u)M = (A + p)m{t) = (A + p)noe0 =t, (7.4.37) 
with M (0) = nå. 

Equation 7.4.37 can be solved exactly using the technique of integrating factors. Its 

solution is X 

M(t) = ne O5) 4 E noet Giza — 1| ; (7.4.38) 

-=p 


From the definition of variance, Equation 6.6.5, the variance of the population in the birth 
and death process equals 


Var[N(t)] = mo EH ome eae = 1| , Aen, (7.4.39) 


or 


II 


Var[N(t)] = 2Anot, A= pL. (7.4.40) 


e Example 7.4.1: Chemical kinetics 


The use of Markov processes to describe birth and death has become quite popular. 
Indeed, it can be applied to any phenomena where something is being created or destroyed. 
Here we illustrate its application in chemical kinetics. 

Let the random variable X(t) be the number of A molecules in a unimolecular reaction 
A — B (such as radioactive decay) at time t. A stochastic model that describes the decrease 
of A can be constructed from the following assumptions: 


1. The probability of transition from n to n — 1 in the time interval (t,t + At) is nAAt + 
o(At) where A is a constant and o(At) denotes that o(At)/At > 0 as At > 0. 


2. The probability of a transition from n to n— j, j > 1, in the time interval (t,t + At) is 
at least o(At) because the time interval is so small that only one molecule undergoes 
a transition. 


3. The reverse reaction occurs with probability zero. 
The equation that governs the probability that X(t) = n is 
prl(t + At) = (n + 1)A\Atpnsi(t) + (1 — AnAt)pn(t) + o(At). (7.4.41) 


Transposing pn(t) from the right side, dividing by At, and taking the limit At —> 0, we 
obtain the differential-difference equation® 


dpn 


aF (n + 1)Apn+1 (t) — nàpn (t). (7.4.42) 


Equation 7.4.42 is frequently called the stochastic master equation. The first term on the 
right side of this equation vanishes when n = no. 


5 McQuarrie, D. A., 1963: Kinetics of small systems. I. J. Chem. Phys., 38, 433-436. 
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The solution of Equation 7.4.42 once again involves introducing a generating function 
for py(t), namely 


no 
F(z t) =X mt), l< (7.4.43) 
n=0 


Summing Equation 7.4.42 from n = 0 to no after multiplying it by z”, we find 


no no—1 no 
3 a rA 2 (n+ 1)2"pn4i(t) — Aare Dn(t). (7.4.44) 
Because 
no 
n@pn _ OF 
oe x at ae (7.4.45) 
n=0 
n a OF 
5 nz” p(t) =z 5 nz”! prn(t) = z—, (7.4.46) 
Oz 
n=0 n=0 
and 
— = k-1 ~ k-1 OF 
n + 1)z”Ppn41 (t) = kz" t) = kz" t)=—, 7.447 
D (m+ Deon) = ket nO =Y taal) = 5 (7.4.47) 


Equation 7.4.44 becomes the first-order partial differential equation 


OF a 
ao 


(7.4.48) 


The solution of Equation 7.4.48 follows the method used to solve Equation 7.4.10. Here 
we find €(z,t) via 
dt dz 


o (7.4.49) 
or 
Elz, t) = (z-—De™. (7.4.50) 
Therefore, 
F(z,t) = f[(z-De™) . (7.4.51) 


To find f(-), we use the initial condition that F(z,0) = z”°. This yields f(y) = (1+y)"° 
and 
F(z,t) = [1+ (¢-1)e"™*]". (7.4.52) 


Once again, we can compute the mean and variance of this process. Because 


OF a 
aan = n(t), 7.4.53 
7| = arn (7.4.53) 
the mean is given by 
OF (1, t) 
E[X(t)] = an (7.4.54) 
To compute the variance, we first compute the second moment. Since 
OR A o, 
z5 5 nz” Pn(t), (7.4.55) 
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and no 
2 (Z) = Donte. (7.4.56) 
we have that oe : 
do palt) B a + a (7.4.57) 


From Equation 6.6.5, the final result is 


Var[ X (t)] (7.4.58) 


_ &F(1,t) OFLE) [OFC t) i 
Oz? Oz Oz i 


Upon substituting Equation 7.4.52 into Equations 7.4.54 and 7.4.58, the mean and variance 
for this process are 


EJX(t)] = noe ~, and Var[ X (t)] = noe™™ (1- e™™) 4 (7.4.59) 


Because the expected value of the stochastic representation also equals the deterministic 
result, the two representations are “consistent in the mean.” Further study shows that this 
is true only for unimolecular reactions. Upon expanding Equation 7.4.52, we find that 


Pr(t) = en amenity, (7.4.60) 


n 


An alternative method to the generating function involves Laplace transforms. To 
illustrate this method, we again examine the reaction A —> B. The stochastic master 
equation is 

dpn 
dt 
pr(t) = 0 for 0 < n < no, where pn(t) denotes the probability that we have n particles of 
B at time t. The initial condition is that pn, (0) = 1 and pm(0) = 0 for m 4 no where no 
denotes the initial number of molecules of B. 
Taking the Laplace transform of Equation 7.4.61, we find that 


= (n — 1)àpn-1 (t) — nApn (t), no Sn < œ, (7.4.61) 


sPa(s) = (n — 1)APa-1 (8) — nAP,(s), no < n < œ, (7.4.62) 


and 
sPao (s) — 1 = —ndPpy (8). (7.4.63) 


Therefore, solving for P, (s), 


n 


_ (n—I1)a eke TN as 
P,,(s) = <a Pr-1(s) = Ger. Ie +k). (7.4.64) 
From partial fractions, 
BCED (=i 
E Ges pa (k — no)!(n — BS + kA) TSE 


6 Ishida, K., 1969: Stochastic model for autocatalytic reaction. Bull. Chem. Soc. Japan, 42, 564-565. 
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Taking the inverse Laplace transform, 


(n—1)! n (—1)*-™(n—n )! ae 
ee (no — 1)!(n — no)! 2 (k= no)!(n= I or (7.4.66) 
2 SAGA CANOE S Ce) th = no)! aie 
~ (no — 1)!(n — no)! = jln — no — j)! (7.4.67) 


(n = 1)!e7ànot a ial 


(no — 1)!(n — no)! (7.4.68) 


2 


where we introduced j = k — no and eliminated the summation via the binomial theorem. 
Equation 7.4.68 is identical to results’ given by Delbriick using another technique. 


e Example 7.4.2 


à 
In the chemical reaction rA 2 B, r molecules of A combine to form one molecule of B. 


H 
If X(t) = n is the number of B molecules, then the probability pn(t) = P{X(t) = n} of 
having n molecules of B is given by 


dpn 

Te = —|nu + (N = n)Al pn + (N =n + 1)àpn-1 + (n + 1) upn+1, (7.4.69) 

where 0 < n < N, rN is the total number of molecules of A, A is the rate at which r 

molecules of A combine to produce B, and p is the rate at which B decomposes into A. 
Multiplying Equation 7.4.69 by z” and summing from n = —1 to N +1, 


N+1 N+1 N+1 N+1 


dpn 
D atema SD omt A-n) Y np NA S Pai 
n=—-1 n=—-1 n=—-1 ==] 
N+1 N+1 
=À 5 (n = 1)” Pn- +u Xo (n + 1)” Pny- (7.4.70) 
n=—1 n=—1 
Defining 
N+1 
Fzt = XO mO, <1, (7.4.71) 
n=—1 


with p_1 = py+1 = 0, we have that 


ðF {H n dpn 
ap NH N l N+1 l 
ag = 5 nz pn = X (i + 1) Pi+1 = 5 (i + 1) Pi+1, (7.4.73) 
n=—1 i=—2 i=—1 


T Delbrück, M., 1940: Statistical fluctuations in autocatalytic reactions. J. Chem. Phys., 8, 120-124. 
See his Equation 7. 
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N+1 
OF n 
y T D nz Pn, (7.4.74) 
OF N+1 N+2 N+1 
: = ntin, = ; Pao io 78 R 
2 a T 2 nz ` Pn = 2 (i — 1) Pi-1 = 2 1)z'Pi-1, (7.4.75) 
and 

N+1 N+2 N+1 
F= S72 pn =D) pias Do ipio (7.4.76) 

not i=0 i=-1 


Therefore, the differential-difference equation, Equation 7.4.70, can be replaced by 


OF 9, OF 
ap NA(z —1)F + [u — (w— Adz — Az") Da" (7.4.77) 
Using the same technique as above, this partial differential equation can be written as 
dt d dF 
= : = ; (7.4.78) 
-1 (l-z)(ut Az) -NA(z-1) 
Equation 7.4.78 yields the independent solutions 
AT? tnt = €(z,t) = constant (7.4.79) 
p+ Az , f 
and 
(u +A)‘ F(z,t) = n(z,t) = another constant, (7.4.80) 


where f(-) is an arbitrary, differentiable function. If there are m units of B at t = 0, 
0< m < N, the initial condition is F(z,0) = 2”. Then, 


l-z ym 
tas) > (u + Az)N’ (7.4.81) 
or 
Fla) = Cie tay (rea) 


After some algebra, we finally find that 


1 m 

= —(u+A)t —(u+A)t 

F(z,t) gry {el e | +2 fà+ne D) 

N-m 

x {u + Ae HAE + dz [1 — ee } : (7.4.83) 
Computing the mean and variance, we obtain 
= me —(u+A)t (N - m)X _ p—(u+A)t 

BE ree [A+ He a re a [1 e ] ; (7.4.84) 


and 
a hH, —(u+A)t — ew (L+A)t 
Var(X) GFN È + pe [1 e | 
(N = m)à 


Gaye [u IF dew Wet] È TE geeta] . (7.4.85) 


Random Processes 375 


Problems 


1. During their study of growing cancerous cells (with growth rate a), Bartoszyński et 
al.8 developed a probabilistic model of a tumor that has not yet metastasized. In their 
mathematical derivation a predictive model gives the probability p,(t) that certain nth 
type of cells (out of N) will develop. This probability can change in two ways: (1) Each of 
the existing cells has the probability AnAt + o(At) of mutating to another type between t 
and t+ At. (2) The probability that cells in state n at time t will shed a metastasis between 
t and t + At is pnce’/“At + o(At), where u is a constant and c is the size of a single cell. 
Setting p = Ac/N and v = uc, the governing equations for p,,(t) are 


dpn 
P = —(p+ vne! py, + p(n + Le! pasa, n= 0, 1,2, oe -N E 1, 
and d 
p a 
a =-(p+v)Ne'/“pn, 


with the initial conditions py (0) = 1 and p,(0) =O ifn AN. 


Step 1: Introducing the generating function 


plz t) =X "p(t, O<2<1, 


show that our system of linear differential-difference equations can be written as the first- 
order partial differential equation 


OF L Ip (p+ v)ajet/a 2? 
Oz 


with $(z,0) = 2%. 
Step 2: Solve the partial differential equation in Step 1 and show that 


(z,t) = (=) fı = (1 = et.) exp[-a(p +v) (e° z Ny. 


Project: Stochastic Simulation of Chemical Reactions 


Most stochastic descriptions of chemical reactions cannot be attacked analytically and 
numerical simulation is necessary. The purpose of this project is to familiarize you with 
some methods used in the stochastic simulation of chemical reactions. In particular, we will 
use the Lokta reactions given by the reaction equations: 


A+X 52X, (1) 
X +Y ŻY, (2) 
Yz (3) 


8 Bartoszyński, R., B. F. Jones, and J. P. Klein, 1985: Some stochastic models of cancer metastases. 
Commun. Statist.-Stochastic Models, 1, 317-339. 
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Figure 7.4.1: The temporal variation of the molecules in a Lokta reaction when At = 1075, kya = 10, 
ko = 0.01, kg = 10, and z(0) = y(0) = 1000. 


Surprisingly, simple numerical integration of the master equation is not fruitful. This occurs 
because of the number and nature of the independent variables; there is only one master 
equation but N reactants and time for independent variables. 

An alternative to integrating the master equation is a direct stochastic simulation. In 
this approach, the (transition) probability for each reaction is computed: pı = k,axAt, 
P2 = koxyAt, and pa = k3yAt, where At is the time between each consecutive state and 
a is the constant number of molecules of A. The obvious question is: Which of these 
probabilities should we use? 

Our first attempt follows Nakanishi:? Assume that At is sufficiently small so that 
pı + p2 + p3 < 1. Using a normalized uniform distribution, such as MATLAB’s rand, 
compute a random variable r for each time step. Then march forward in time. At each 
time step, there are four possibilities. If 0 < r < pı, then the first reaction occurs and 
x(t + At) = x(t) +1, y(t + At) = y(t). If pı <r < pı + po, then the second reaction occurs 
and z(t + At) = x(t) — 1, y(t + At) = y(t)+ 1. If pı +p2 < r < pı + p2 + p3, then the third 
reaction occurs and a(t + At) = x(t), y(t + At) = y(t) — 1. Finally, if pı + p2 +p3 <r <1, 
then no reaction occurs and x(t + At) = z(t), y(t + At) = y(t). 

For the first portion of this project, create MATLAB code to simulate our chemical 
reaction using this simulation technique. Explore how your results behave as you vary x(0), 
y(0) and especially At. See Figure 7.4.1. 

One of the difficulties in using Nakanishi’s method is the introduction of At. What 
value should we choose to ensure that pı + po + p3 < 1? Several years later, Gillespie! 


9 This is the technique used by Nakanishi, T., 1972: Stochastic analysis of an oscillating chemical 
reaction. J. Phys. Soc. Japan, 32, 1313-1322. 


10 Gillespie, D. T., 1976: A general method for numerically simulating the stochastic time evolution 
of coupled chemical reactions. J. Comput. Phys., 22, 403-434; Gillespie, D. T., 1977: Exact stochastic 
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Figure 7.4.2: Same as Figure 7.4.1 except that Gillespie’s method has been used. 


developed a similar algorithm. He introduced three parameters, a, = ky, ax, a2 = koxy, and 
a3 = k3y, along with ap = a, + a2 + a3. These parameters a1, a2, and ag are similar to the 
probabilities p1, p2, and p3. Similarly, he introduced a random number rə that is chosen 
from a normalized uniform distribution. Then, if 0 < reao < a1, the first reaction occurs 
and «(t+ At) = x(t) +1, y(t + At) = y(t). If a1 < reap < a1 + a2, then the second reaction 
occurs and #(t+ At) = x(t)— 1, y(t+ At) = y(t) +1. If ay +a2 < reao < ao, then the third 
reaction occurs and z(t + At) = z(t), y(t + At) = y(t) — 1. Because of his selection criteria 
for the reaction that occurs during a time step, one of the three reactions must take place. 
See Figure 7.4.2. 

The most radical difference between the Nakanishi and Gillespie schemes involves the 
time step. It is no longer constant but varies with time and equals At = In(1/r1)/ao, where 
rı is a random variable selected from a normalized uniform distribution. The theoretical 
justification for this choice is given in Section III of his paper. 

For the second portion of this project, create MATLAB code to simulate our chemical 
reaction using Gillespie’s technique. You might like to plot z(t) vs y(t) and observe the 
patterns that you obtain. 

Finally, for a specific time, compute the probability density function that gives the 
probability that x and y molecules exist. See Figure 7.4.3. 


7.5 POISSON PROCESSES 


The Poisson random process is a counting process that counts the number of occur- 
rences of some particular event as time increases. In other words, for each value of t, there 


simulation of coupled chemical reactions. J. Phys. Chem., 81, 2340-2361 
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Figure 7.4.3: The estimated probability density function for the chemical reactions given by Equations 
(1) through (3) (for X on the left, Y on the right) at time t = 10. Five thousand realizations were used in 
these computations. 


is a number N (t), which gives the number of events that occurred during the interval (0, t]. 
For this reason N (t) is a discrete random variable with the set of possible values {0, 1, 2,...}. 
Figure 7.5.1 illustrates a sample function. We can express this process mathematically by 


N(t) = X H(t-T[nl), (7.5.1) 
n=0 


where Tfn] is the time to the nth arrival, a random sequence of times. The question now 
becomes how to determine the values of T[n]. The answer involves three rather physical 
assumptions. They are: 


1. N(0) =0. 


2. N(t) has independent and stationary increments. By stationary we mean that for any 
two equal time intervals At; and Atz, the probability of n events in At, equals the 
probability of n events in Atz. By independent we mean that for any time interval 
(t,t + At) the probability of n events in (t,t + At) is independent of how many events 
have occurred earlier or how they have occurred. 


3. 
1— AAT, k=0, 
P[N(t + At) — N(t) =k] = AAt, k=1, (7.5.2) 
0, k>1, 


for all t. Here \ equals the expected number of events in an interval of unit length of 
time. Because E[N(t)] = A, it is the average number of events that occur in one unit 
of time and in practice it can be measured experimentally. 


We begin our analysis of Poisson processes by finding P|N(t) = 0] for any t > 0. If 
there are no arrivals in [0, t], then there must be no arrivals in [0,t— At] and also no arrivals 
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Figure 7.5.1: Schematic of a Poisson process. 


in (t — At, t]. Therefore, 


Because N (t) is independent, 


PINŒ =0] = P[N(t— At) = 0]P[N(t) — N(t — At) = 0]. 


Furthermore, since N(t) is stationary, 


P(N(t) = 0] = P[N(t — At) = 0]P[N(t + At) — N(t) = 0]. 


Finally, from Equation 7.5.2, 
P|N(t) = 0] = P[N(t — At) = 0](1 — AAD). 
Let us denote P[N(t) = 0] by Po(t). Then, 
Po(t) = Po(t — At)(1 — AAD), 
Po(t) — Po(t — At) 


At 
Taking the limit as At > 0, we obtain the (linear) differential equation 


= —\Po(t — At). 


dPo(t) 
dt 


The solution of Equation 7.5.9 is 
Po(t) = Ce, 
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(7.5.3) 


(7.5.4) 


(7.5.5) 


(7.5.6) 


(7.5.7) 


(7.5.8) 


(7.5.9) 


(7.5.10) 


where C is an arbitrary constant. To evaluate C, we have the initial condition P9(0) = 


P[N(0) =0] = 1 from Axion 1. Therefore, 


PING) = 0] = P(t) e 


(7.5.11) 
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Next, let us find P, (t) = P[N(t) = 1]. We either have no arrivals in [0,¢— At] and one 
arrival in (t — At, t] or one arrival in [0,t — At] and no arrivals in (t — At, t]. These are the 
only two possibilities because there can be at most one arrival in a time interval At. The 
two events are mutually exclusive. Therefore, 


P[N(t) =1] = P[N(t— At) = 0, N(t) — N(t— At) = 1] 
+ PING- At) = 0, N(t) — N(t— At) = 0] (7.5.12) 
= P[N(t— At) = 0]P[N(t) — N(t— At) = 1] 
+ P[N(t— At) = 1]P[N(t) — N(t— At) =0] (7.5.13) 
= P[N(t — At) =0]P[N(t + At) — N(t) = 1] 
+ PING- At) = 1]P[N(t + At) — N(t) = 0]. (7.5.14) 


Equation 7.5.13 follows from independence while Equation 7.5.14 follows from stationarity. 
Introducing P; (t) in Equation 7.5.14 and using Axion 3, 


P,(t) = Po(t — At)AAt + Pi (t — At)(1 — AAA), (7.5.15) 
a RO- (t—A 
ie) = Att za AP, (t — At) + AP(t — At). (7.5.16) 
Taking the limit as At > 0, we obtain 
P(t 
ari) + \Pi(t) = APA(t). (7.5.17) 
In a similar manner, we can prove that 
Py(t 
a bay NP, (t) = APy_1(t), (7.5.18) 


where k = 1,2,3,... and P(t) = P[N(t) = k]. 
This set of simultaneous linear equations can be solved recursively. Its solution is 


(At) 
kl’ 


P(t) = exp(—At) k=O 1 Diva ts (7.5.19) 
which is the Poisson probability mass function. Here is the average number of arrivals 
per second. 

In the realization of a Poisson process, one of the important quantities is the arrival 
time, tn, shown in Figure 7.5.1. Of course, the arrival time is also a random process and 
will change with each new realization. A related quantity Z; = ti — t;_1, the time intervals 
between two successive occurrences (interoccurrence times) of Poisson events. We will now 
show that the random variables Z1, Z2, etc., are independent and identically distributed 
with 

P(Za < x£) =1-e7*%, z>0, n=1,2,3,... (7.5.20) 


We begin by noting that 


P(Z, >t) = P[N(t) = 0] = e~ (7.5.21) 
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from Equation 7.5.19. Therefore, Z, has an exponential distribution. 

Let us denote its probability density by pz,(z1). From the joint conditional density 
function, 

&1 
P(Zə > t) = P(Zə > t|Z1 = 21)pz, (21) dz, (7.5.22) 
0 

where 0 < ¿& < t. If Zi = z1, then Zo > t if and only if N(t + z1) — N(z,) = 0. Therefore, 
using the independence and stationary properties, 


P{Zo > t|Z, = P[N(t+ 21) — N(a1) = 0]} = PIN(t) = 0] =e. (7.5.23) 


Consequently, 
P(Z2 >t) =e, (7.5.24) 


showing that Zə is also exponential. Also, Zə is independent of Zı. Now, let us introduce 
PZ(Z2) as the probability density of Zı + Z2. By similar arguments we can show that Z3 
is also exponential. The final result follows by induction. 


e Example 7.5.1: Random telegraph signal 


We can use the fact that interoccurrence times are independent and identically dis- 
tributed to realize the Poisson process. An important application of this is in the generation 
of the random telegraph signal: X(t) = (—1)N®©. However, no one uses this definition to 
compute the signal; they use the arrival times to change the signal from +1 to —1 or vice 
versa. 

We begin by noting that T; = T;-1 + Zi, with i = 1,2,..., To = 0, and T; is the ith 
arrival time. Each Z; has the same exponent probability density function. From Equation 
6.4.17, 

— ži a 7.5.25) 
pom ti) Te (7.5. 
where the U;’s are from a uniform distribution. The realization of a random telegraphic 
signal is given by the MATLAB code: 


clear 
N = 100; % number of switches in realization 
lambda = 0.15; % switching rate 
xX=[]; 
% generate N uniformly distributed random variables 
S = rand(1,N); 
% transform S into an exponential random variable 
T = - log(S)/lambda; 
V = cumsum(T); % compute switching times 
= [0.01:0.01:100]; % create time array 
icount = 1; amplitude = -1; % initialize X(t) 
for k = 1:10000 
if ( t(k) >= V(icount) ) % at each switching point 
icount = icount + 1; 
amplitude = - amplitude; % switch sign 
end 
X(k) = amplitude; % generate X(t) 
end 


ct 
I 
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plot(t,X) % plot results 
xlabel(’\it t’,’FontSize’ ,25) ; 
ylabel(’\it X(t)/a’,’FontSize’ ,25); 
axis([0 max(t) -1.1 1.1]) 


This was the MATLAB code that was used to generate Figure 7.5.2. 


e Example 7.0.2 


It takes a workman an average of one hour to put a widget together. Assuming that 
the task can be modeled as a Poisson process, what is the probability that a workman can 
build 12 widgets during an eight-hour shift? 

The probability that n widgets can be constructed by time t is 


(At)” 


n! 


P[N(t) =n] = e~ (7.5.26) 


Therefore, the probability that 12 or more widgets can be constructed in eight hours is 


PING) >12 =e X 2 = 0.1119, (7.5.27) 
n. 


n=12 


since À = 1. 
We could have also obtained our results by creating 12 exponentially distributed time 
periods and summed them together using MATLAB: 


t uniform = rand(1,12); 
T = - log(1-t_uniform) ; 
total_time = sum(T); 


Then, by executing this code a large number N of times and counting the number icount 
of times that total_time <= 8, the probability equals icount / N. 


Problems 
1. Use the generating function 
F(z,t) = N pn(t)2”, |z| <1, 
n=0 


with F(z,0) = 1 to solve Equation 7.5.18 by showing that F(z,t) = e“4-). Then, by 
expanding F'(z,t), recover Equation 7.5.19. 


Random Processes 383 


Figure 7.5.1: The probability density P(y) of the output from an ideal integrator with finite memory 
when the input is a random telegraphic signal when At = 0.01, A = 2, and 7, = 10. 


Project: Output from a Filter 
When the Input Is a Random Telegraphic Signal !! 


In the study of many systems, such as linear filters, the output y(-) can be written as 


y(t) = L W(t — r)a(r) dr, 


where W(-) is the weight function and z(-) is the input. The purpose of this project is 
to explore the probability density P(y) of the output when z(t) is the random telegraphic 
signal, a Poisson random process. You will filter this input two ways: (1) ideal integrator 
with finite memory: W(t) = H(t) — H(t — 71), m > 0, and (2) simple RC = 1 low-pass 
filter W(t) =e‘ H(t). 


Step 1: Use MATLAB to code z(t) where the expected time between the zeros is À. 
Step 2: Develop MATLAB code to compute y(t) for each of the weight functions W(t). 
Step 3: Compute P(y) for both filters. How do your results vary as A varies? 


Further Readings 


Beckmann, P., 1967: Probability in Communication Engineering. Harcourt, Brace & World, 
511 pp. A presentation of probability as it applies to problems in communication engineer- 
ing. 


Gillespie, D. T., 1991: Markov Processes: An Introduction for Physical Scientists. Academic 
Press, 592 pp. For the scientist who needs an introduction to the details of the subject. 


Hsu, H., 1997: Probability, Random Variables, & Random Processes. McGraw-Hill, 306 pp. 
Summary of results plus many worked problems. 


11 Suggested by a paper by McFadden, J. A., 1959: The probability density of the output of a filter when 
the input is a random telegraphic signal: Differential-equation approach. IRE Trans. Circuit Theory, 6, 
228-233. 
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Figure 7.5.2: The probability density P(y) of the output from a simple RC = 1 filter, when the input is 
a random telegraphic signal, when à = 1, and At = 0.05. 


Kay, S. M., 2006: Intuitive Probability and Random Processes Using MATLAB. Springer, 
833 pp. A well-paced book designed for the electrical engineering crowd. 


Ross, S. M., 2007: Introduction to Probability Models. Academic Press, 782 pp. An intro- 
ductory undergraduate book in applied probability and stochastic processes. 


Tuckwell, H. C., 1995: Elementary Applications of Probability Theory. Chapman & Hall, 
292 pp. This book presents applications using probability theory, primarily from biology. 


Chapter 8 
Ito’s Stochastic Calculus 


In elementary differential equation classes, students study the solution to first-order 
ordinary differential equations 


dx 


Pig a(t, £), x(0) = xo. (8.0.1) 


There we showed that Equation 8.0.1 has the solution 


t 
x(t) = x(0) +f aln, x(y)| dn. (8.0.2) 
0 
Consider now the analogous stochastic differential equation: 
dX (t) = alt, X(t)] dt, X(0) = Xo. (8.0.3) 


Although Equations 8.0.1 and 8.0.3 formally appear the same, an immediate question is 
what is meant by dX(t). In elementary calculus, the concept of the infinitesimal involves 
limits, continuity, and so forth. As we shall see in Section 8.2, Brownian motion, a very 
common stochastic process, is nowhere differentiable. Here we can merely say that dX (t) = 
X(t + dt) — X(t). 

Consider now a modification of Equation 8.0.3 where we introduce a random forcing: 


dX(t) = alt, X(t)] dt + bft, X(t)|dB(t), X(0) = Xo. (8.0.4) 


Here dB(t) = B(t + dt) — B(t), B(t) denotes Brownian motion and aft, X (t)] and bft, X(t)] 
are deterministic functions. Consequently, changes to X(t) result from (1) the effects of the 
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initial conditions and (2) noise generated by Brownian motion (the driving force). Stochastic 
processes governed by Equation 8.0.4 are referred to as It6 processes. 

Following the methods used to derive 8.0.2, we can formally write the solution to 
Equation 8.0.4 as 


de ee i alm X(n)] dn + I bin, X(n)] dB(n). (8.0.5) 


The first integral in Equation 8.0.5 is the conventional Riemann integral from elementary 
calculus and is well understood. The second integral, however, is new and must be treated 
with care. It is called It6’s stochastic integral and treated in Section 8.3. 

In summary, a simple analog to first-order ordinary differential equations for a single 
random variable X (t) raises several important questions. What is meant by the infinitesimal 
and the integral in stochastic calculus? In this chapter we will focus on It6 processes and 
the associated calculus. Although It6’s calculus is an important discipline, it is not the only 
form of stochastic calculus. The interested student is referred elsewhere for further study. 


Problems 


1. The Poisson random process N (t) is defined by 


N(t) = 5 H(t — tn), 


n=1 


where t, is a sequence of independent and identically distributed inter-arrival times tn. 
A graphical representation of N(t) would consist of ever-increasing steps with the edges 
located at t = tn. Use the definition of dN(t) = N(t + dt) — N (t) to show that 


1 for t = tn, 


otherwise. 


2. The telegraph signal is defined by X(t) = (—1)N®, where N(t) is given by the Poisson 
random distribution in Problem 1. Show! that 


dX(t) = X(t + dt) — X(t) = (1) ® K NO — 1| = -2X (4) dN (2). 
Hint: Consider dN (t) at various times. 
3. If X(t) and Y(t) denote two stochastic processes, use the definition of the derivative to 


show that (a) d[cX(t)] = cdX (t), where cis a constant, (b) d[X (t) £Y (t)] = dX (t) +dY (t), 
and (c) d[X(t)Y (t)] = X(t) dY (t) + Y(t) dX (t) + dX (t) dY (t). 


8.1 RANDOM DIFFERENTIAL EQUATIONS 


A large portion of this book has been devoted to solving differential equations. Here 
we examine the response of differential equations to random forcing where the differential 


1 Taken from Janaswamy, R., 2013: On random time and on the relation between wave and telegraph 
equation. IEEE Trans. Antennas Propag., 61, 2735-2744. 
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equation describes a nonrandom process. This is an important question in the sciences and 
engineering because noise, a random phenomenon, is ubiquitous in nature. 

Because the solution to random differential equations can be found by conventional 
techniques, we can use them to study the effect of randomness on the robustness of a 
solution to a differential equation subject to small changes of the initial condition. Although 
this may be of considerable engineering interest, it is really too simple to develop a deep 
understanding of stochastic differential equations. 


e Example 8.1.1: LR circuit 


One of the simplest differential equations involves the mathematical model for an LR 
electrical circuit: 
dI 


L% + RI = E(t), (8.1.1) 


where I(t) denotes the current within an electrical circuit with inductance L and resistance 
R, and E(t) is the mean electromotive force. If we solve this first-order ordinary differential 
equation using an integrating factor, its solution is 


Ht) = 10 ( +) | zep( mn [re ) ep ) ap (8.1.2) 


Clearly, if the electromotive forcing is random, so is the current. 

In the previous chapter we showed that the mean and variance were useful parameters 
in characterizing a random variable. This will also be true here. If we find the mean of the 
solution, 


Rt 
E|I(t)] = I (0) exp (-=) (8.1.3) 
provided E[F(t)] = 0. Thus, the mean of the current is the same as that for an ideal LR 


circuit. 
Turning to the variance, 


ox(t) = EP H] - {EUO (8.1.4) 


= E[ PO ex(- =) 

a 

a = S I E|F Fe =F) dr! dr (8.1.5) 
-PO ep(- *) 


= exp(- my I EIF Ee AES dr! dr. (8.1.6) 


To proceed further we need the autocorrelation E|F(r)F(r’)]. In papers by Ornstein 
et al.? and Jones and McCombie,? they adopted a random process with the autocorrelation 


2 Ornstein, L. S., H. C. Burger, J. Taylor, and W. Clarkson, 1927: The Brownian movement of a 
galvanometer and the influence of the temperature of the outer circuit. Proc. Roy. Soc. London, Ser. A, 
115, 391-406. 


3 Jones, R. V., and C. W. McCombie, 1952: Brownian fluctuations in galvanometer and galvanometer 
amplifiers. Phil. Trans. Roy. Soc. London, Ser. A, 244, 205-230. 
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function 


E|F(r)F(r’)| = 2D6(7 — 7’). (8.1.7) 


The advantage of this process is that it is mathematically the simplest because it possesses a 
white power spectrum. Unfortunately this random process can never be physically realized 
because it would possess infinite mean square power. All physically realizable processes 
involve a power spectrum that tends to zero at sufficiently high frequencies. If ®(w) denotes 
the power spectrum, this condition can be expressed as 


j (w) dw < oo. (8.1.8) 
0 
In view of these considerations, let us adopt the autocorrelation 
Rx(t-7')= | P(w) cos[w(t — T')] dw, (8.1.9) 
0 


where ®(w) is the power spectrum of F(T). Therefore, the variance becomes 


o%( =al px z)exp| At) e| PUD] cosul = ree dra! 
(8.1.10) 


Reversing the ordering of integration, 


Pas af x o) f f ev- ROAT] costul ~ 1] dr dr dw. (8.1.11) 


We can evaluate the integrals involving 7 and 7’ exactly. Equation 8.1.11 then becomes 


L w) 2 tz 
ox (t) =| w+ RJT [1 h eT?RtL Be BY” cos(wt)| dw. (8.1.12) 


Let us now consider some special cases. As t — 0, 0%(t) —> 0 and the variance is 
initially small. On the other hand, as t > oo, 


o% (t) = [ ae dw. (8.1.13) 


Thus, the variance grows to a constant value, which we would have found by using Fourier 
transforms to solve the differential equation. 

Consider now the special case P(w) = 2D/n, a forcing by white noise. Ignoring the 
defects in this model, we can evaluate the integrals in Equation 8.1.13 exactly and find that 


DL 
ox (t) = (1 = ened) (8.1.14) 


These results are identical to those found by Uhlenbeck and Ornstein* in their study of a 
free particle in Brownian motion. 


4 Uhlenbeck, G. E., and L. S. Ornstein, 1930: On the theory of the Brownian motion. Phys. Review, 
36, 823-841. See the top of their page 828. 
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e Example 8.1.2: Damped harmonic motion 


Another classic differential equation that we can excite with a random process is the 
damped harmonic oscillator: 


y" + kwoy’ + wey = F(t), (8.1.15) 
where 0 < £ < 1, y denotes the displacement, t is time, wi = k/m, 2£wo = B/m, m is 


the mass of the oscillator, k is the linear spring constant, and 8 denotes the constant of a 
viscous damper. The solution to this second-order ordinary differential equation is 


1 0 t 
ult) = yOer" [cos(ert) + 28 sinon) | + est sinat) + [alt= ar, 
1 1 0 
(8.1.16) 
where wı = woy 1 — €?, and 
e Swot 
h(t) = sin(w t) H(t). (8.1.17) 
Wy 
Again we begin by finding the mean of Equation 8.1.16. It is 
‘(0 
Ely(t)] = y(O)e S08 fosco + sao sin(wxt)| +O) t sin(w t)+ +f h(t—7)E|F(7)] dr. 
1 1 
n 1.18) 
If we again choose a random process where E[F(t)] = 0, the integral vanishes and the 


stochastic mean of the motion only depends on the initial conditions. 
Turning to the variance, 


o3 (t) = Ely? (t)| — {Ely(t)|}? = i [ h(t — r)h(t — 7’) E[F(r)F(7’)| dr dr’. (8.1.19) 
If we again adopt the autocorrelation function 
Rx(t-7T')= i P(w) coslw(T — 7’)] dw, (8.1.20) 
where ®(w) is the power spectrum of F(T), then 


ey = f° 26 


Carrying out the integrations in 7 and 7’, we finally obtain 


t 
e7 $40 (2t—-7—7') sin[w1(t—7)] sinw1(t—7’)] cos[w(7r —7')] dr dr’ dw. 


(8.1.21) 


o3 = "RU e7 ent 260 in(w os(w 
o= | WOE (1+ T sin(w1t) cos(w1t) 


2 2 
— ewost E cos(wıt) + gwo snort cos(wt) — efwot 2 sin(w t) sin(wt) 
Wy Wy 


2.2 2 2 
Seo are sin?(wrt)} ) dw, (8.1.22) 


wI 


where |Q(w)|? = (we — w?)? + 4w?wé?. 
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Figure 8.1.1: The mean and variance of the response for the differential equation y’ + y = f(t) when 
forced by Gaussian random noise. The parameters used are y(0) = 1 and Ar = 0.01. 


As in the previous example, o3% (t) + 0 as t > 0 and the variance is initially small. 
The steady-state variance now becomes 


24,_ f[ 2) 4. 
v(t) = f Ma mar? (8.1.23) 


Finally, for the special case P(w) = 2D/z, the variance is 


D —2Ewot 
o? (t) = Ju? fı 7” [wy + wow € sin(2w,t) + 2wa sin*(wxt))} : (8.1.24) 
0 1 


These results are identical to those found by Uhlenbeck and Ornstein® in their study of a 
harmonically bound particle in Brownian motion. 


Project: Low-Pass Filter with Random Input 
Consider the initial-value problem 
y +y= f(t), y(0) = yo. 
It has the solution P 
y(t) = yoe ™* + a e f(r) dr. 
This differential equation is identical to that governing an RC electrical circuit. This circuit 


has the property that it filters out high-frequency disturbances. Here we explore the case 
when f(t) is a random process. 


5 Ibid. See their pages 834 and 835. 
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Figure 8.1.2: The probability density function for the response to the differential equation y’ + y = f(t) 
when f(t) is a Gaussian distribution. Twenty thousand realizations were used to compute the density 
function. Here the parameters used are y(0) = 0 and Ar = 0.01. 


Step 1: Using the MATLAB intrinsic function randn, generate a stationary white noise 
excitation of length N. Let deltat denote the time interval At between each new forcing so 
that n = 1 corresponds to t = 0 and n = N corresponds to the end of the record t = T. 


Step 2: Using the Gaussian random forcing that you created in Step 1, develop a MATLAB 
code to compute y(t) given y(0) and f(t). 


Step 3: Once you have confidence in your code, modify it so that you can generate many 
realizations of y(t). Save your solution as a function of t and realization. Use MATLAB’s 
intrinsic functions mean and var to compute the mean and variance as a function of time. 
Figure 8.1.1 shows the results when 2000 realizations were used. For comparison the mean 
and variance of the forcing have also been included. Ideally this mean and variance should 
be zero and one, respectively. We have also included the exact mean and variance, given 
by Equation 8.1.3 and Equation 8.1.14, when we set L = R = 1 and D = At/2. 


Step 4: Now generalize your MATLAB code so that you can compute the probability density 
function of finding y(t) lying between y and y + dy at various times. Figure 8.1.2 illustrates 
four times when y(0) = 0 and Ar = 0.01. 


Step 5: Modify your MATLAB code so that you can compute the autocovariance. See Figure 
8.1.3. 


Project: First-Passage Problem with Random Vibrations © 


In the design of devices, it is often important to know the chance that the device will 
exceed its design criteria. In this project you will examine how often the amplitude of a 


6 Based on a paper by Crandall, S. H., K. L. Chandiramani, and R. G. Cook, 1966: Some first-passage 
problems in random vibration. J. Appl. Mech., 33, 532-538. 
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autocovariance 


Figure 8.1.3: The autocovariance function for the differential equation y’ + y = f(t) when f(t) is a 
Gaussian distribution. Twenty thousand realizations were used. The parameters used here are y(0) = 0 
and Ar = 0.01. 


simple, slightly damped harmonic oscillator 
y” + UMwoy’ +ugy= f(t), 0<¢<«1, (8.1.25) 


will exceed a certain magnitude when forced by white noise. In the physical world this 
transcending of a barrier or passage level leads to “bottoming” or “short circuiting.” 


Step 1: Using the MATLAB command randn, generate a stationary white noise excitation 
of length N. Let deltat denote the time interval At between each new forcing so that n = 
1 corresponds to t = 0O and n = N corresponds to the end of the record t = T. 


Step 2: The exact solution to Equation 8.1.25 is 


y(t) = y(O)e S08 eos VIF + ere sin( VT Gon) 


p20) eut sin( VI- Gant) (8.1.26) 
N f whens sin yI F wolt — r)| f(r) 
0 


z d(woT) 


YO) -wot sin ( VI= C wot) (8.1.27) 
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Figure 8.1.4: A realization of the random function y(t) governed by Equation (1) when forced by the 
Gaussian random forcing shown in the top frame. The parameters used here are y(0) = 1, y’(0) = 0.5, 
¢ = 0.1, and woAr = 0.02. 


cos( y1 —¢ wot) 

/1 — ¢? 
Because you will be computing numerous realizations of y(t) for different f(t)’s, an efficient 
method for evaluating the integrals must be employed. Equation 8.1.27 is more efficient 
than Equation 8.1.26. 

Using the Gaussian random forcing that you created in Step 1, develop a MATLAB code 
to compute y(t) given y(0), y’(0), ¢ and f(t). Figure 8.1.4 illustrates a realization where 
the trapezoidal rule was used to evaluate the integrals in Equation 8.1.27. 


e Swot 


l eS vor sin( V1- (wor) n d(woT). 
0 


0 


Step 3: Now that you can compute y(t) or y(n) for a given Gaussian random forcing, 
generalize your code so that you can compute irun realizations and store them in y(n,m) 
where m = 1:irun. For a specific n or wot, you can use MATLAB’s commands mean and var 
to compute the mean px(t) and the variance oł (t). Figure 8.1.5 shows the results when 
1000 realizations were used. For comparison the mean and variance of the forcing have also 
been included. Ideally this mean and variance should be zero and one, respectively. The 
crosses give the exact results that 


x(t) = y(O)e~$#" tos v1 = Gant + ae so VI= Bont) 


1-2 


YAO): eua ay 
tae c ‘sin( 1= Punt) 


and Equation 8.1.24 when D = woAt/2. 


Step 4: Finally, generalize your MATLAB code so that you store the time T(m) that the 
solution y(n) exceeds a certain amplitude b > 0 for the first time during the realization m. 
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Figure 8.1.5: The mean py (t) and variance oł (t) of a slightly damped simple harmonic oscillator when 


forced by the Gaussian random noise. The parameters used here are y(0) = 1, y/(0) = 0.5, Ç = 0.1, and 
woAr = 0.02. 


Of course, you can do this for several different b’s during a particular realization. Once you 
have this data you can estimate the probability density function using histc. Figure 8.1.6 
illustrates four probability density functions for b = 0.4, b = 0.8, b= 1.2, and b = 1.6. 


Project: Wave Motion Generated by Random Forcing” 


In the previous projects we examined ordinary differential equations that we forced 
with a random process. Here we wish to extend our investigation to the one-dimensional 
wave equation 


u u 

L- n t)d[x — X(t 

Sn Ga = cos(wt) blir — X (6), 
subject to the boundary conditions 

lim u(x,t) > 0, 0<t, 
|z| co 
and initial conditions 
u(x, 0) = u(x,0) = 0, —0 < T < 0. 


Here w is a constant and X(t) is a stochastic process. 
In Example 5.4.4 we show that the solution to this problem is 


u(x,t) = a H[t — r — |X (T) — x|] cos(wr) dr. 


7 Based on a paper by Knowles, J. K., 1968: Propagation of one-dimensional waves from a source in 
random motion. J. Acoust. Soc. Am., 43, 948-957. 
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Figure 8.1.6: The probability density function that a slightly damped oscillator exceeds b at the time 
woT. Fifty thousand realizations were used to compute the density function. The parameters used here are 
y(0) = 0, y’(0) = 0, ¢ = 0.05, and wo Ar = 0.05. The mean value of woT is 10.7 when b = 0.4, 41.93 when 
b = 0.8, 188.19 when b = 1.2, and 1406.8 when b = 1.6. 


When the stochastic forcing is absent X(t) = 0, we can evaluate the integral and find that 


Wee = Hl jal \eintute ie. 


Step 1: Invoking the MATLAB command randn, use this Gaussian distribution to numeri- 
cally generate an excitation X(t). 


Step 2: Using the Gaussian distribution from Step 1, develop a MATLAB code to com- 
pute u(x,t). Figure 8.1.7 illustrates one realization where the trapezoidal rule was used to 
evaluate the integral. 


Step 3: Now that you can compute u(x,t) for a particular random forcing, generalize your 
code so that you can compute irun realizations. Then, for particular values of x and t, you 
can compute the corresponding mean and variance from the irun realizations. Figure 8.1.8 
shows the results when 10,000 realizations were used. 


Step 4: Redo your calculations but use a sine wave with random phase: X(t) = Asin(Qt+6), 
where A and Q are constants and € is a random variable with a uniform distribution on 
(0, 27]. 


8.2 RANDOM WALK AND BROWNIAN MOTION 
In 1827 the Scottish botanist Robert Brown (1773-1858) investigated the fertilization 


process in a newly discovered species of flower. Brown observed under the microscope that 
when the pollen grains from the flower were suspended in water, they performed a “rapid 
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Figure 8.1.7: The solution (realization) of the wave equation when forced by a Gaussian distribution and 
w = 2. In frame (a), there is no stochastic forcing X(t) = 0. Frame (b) shows one realization. 


variance 


Figure 8.1.8: The mean and variance when the wave equation is forced by the stochastic forcing cos(wt)6[x 
—X(t)], where w = 2 and X(t) is a Gaussian distribution. 


oscillation motion.” This motion, now known as Brownian motion, results from the random 
kinetic strikes on the pollen grain by water molecules. Brownian motion is an example of 
a random process known as random walk. This process has now been discovered in diverse 
disciplines, from biology® to finance. In this section we examine its nature. 

Consider a particle that moves along a straight line in a series of steps of equal length. 
Each step is taken, either forwards or backwards, with equal probability Z. After taking 
N steps, the particle could be at any one (let us denote it m) of the following points: 


8 Codling, E. A., M. J. Plank, and S. Benhamou, 2008: Random walk models in biology. J. R. Soc. 
Interface, 5, 813-834. 
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Figure 8.2.1: Three realizations of a one-dimensional random walk where N = 50. 


N,—N +1,...,—1,0,1,...,N — 1 and N. Here m is a random variable. 
We can generate realizations of one-dimensional Brownian motion using the MATLAB 
code: 


clear 


NN = 50; % select the number of steps for the particle to take 
t = (0:1:NN); % create ‘‘time’’ as the particle moves 


% create an array to give the position at each time step 
m = zeros(size(t)); 
m(1) = 0; 4% initialize the position of particle 


for N = 1:NN % now move the particle 
x = rand(1); % generate a random variable lying between [0,1] 
if (x <= 0.5) step = 1; % if less then 0.5, make it a ‘‘head’’ 
else step = -1; end % otherwise it is a ‘‘tail’’ 

% move the particle one step to the right or left 
m(N+1) = m(N) + step; 


end 
% plot the results 
hold on 


plot(t,m,’--ko’,’LineWidth’ ,2,’MarkerSize’ ,8) 
xlabel(’N’,’FontSize’,25); ylabel(’m’,’FontSize’ ,25) 
grid on % add a grid to axes 


Figure 8.2.1 illustrates three such realizations. 
A natural question would now be: What are the quantitative properties of random 
walk? In particular, what is the probability P(m, N) that the particle is at point m after N 


displacements? We begin by noting the probability of any given sequence of N steps is (4) i 
The desired probability P(m, N) equals (4) N times the number of distinct sequences of steps 
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that will lead to the point m after N steps. To reach m, we must take (N+m)/2 steps in the 
positive direction and (N—m)/2 in the negative direction since (N-+m)/2—(N—m)/2 = m. 
(Note both m and N must be even or odd.) The number of these distinct sequences is 
N! 
Every Oya 


(8.2.1) 


Therefore, ; 
N! 1 
|= [E(N +m)]! [i (N - m)|! (5) (8.2.2) 


Comparing these results with Equation 6.6.14, we see that P(m, N) is simply a binomial 
distribution. For this reason, we immediately know E(m) = 0 and Var(m) = N. That 
is, the average position is the origin and the spread of the Brownian motion occurs as the 
square root of steps taken increases. 

The case of greatest interest arises when N is large and m < N. Then we can approx- 
imate P(m, N) by the Poisson distribution, 


P(m, N) x Vel) (8.2.3) 


Let us reexpress Equation 8.2.3 in terms of xz and t where z = mAg and t = NAt. Using 
these definitions, our equation becomes 


P(m, N 


P(x,t) 


: ( 2 ) (8.2.4) 
= ex j .2. 
2vVrDt ”\ 4Dt 
if we define D = (Ax)? /(2At). The attentive student will note that P(x, t) is the Green’s 
function for the heat function, Example 5.5.1. 

An alternative approach to this problem would be to compute many random walks and 
then calculate the probability density function from these computations. We can construct 
a MATLAB code to do this. First we would realize many random walks (here 2000) and 
count the number of times that they end at position m: 


clear 


NN = 100; % set the end point of the random walks 

% introduce intermediate positions along the random walk 
t = (0:1:NN); 

% initialize array 
m = zeros(size(t)); 


‘‘m?? which gives the position at any time 


for icount = 1:2000 % now perform many random walks 
m(1) = 0; % initial position of particle in each walk 


for N = 1:NN 
x = rand(1); % create a random variable between (0, 1] 
% if ’?’x’? less than 0.5, we have a ’’heads’’ 
if (x <= 0.5) step = 1; 
else step = -1; end % otherwise we have a ’’tail’’ 
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Figure 8.2.2: Numerical computation of P(m, 100) using 2000 random walks. The black line gives Equation 
8.2.3. 


m(N+1) = m(N) + step; % now take a step forward or backward 
end 


% set up array that tracks of the final position of the particle 
location(icount) = m(N+1); 


end 


xx = -40:1:40; 

% now count the particles that ended somewhere 

% between -40 and 40 

[n,xout] = hist(location,xx) 

% for comparison, compute Equation 8.2.3 

w-exact = sqrt(2/(pi*NN) )*exp(-xout.*xout/(2*NN) ) ; 

n =n / 2000; % now compute the mass probability function 


% plot the results 

bar_h = bar(xout,n) 

bar_child = get(bar_h, ’Children’ ) 

set (bar_child, ’CData’ ,n) 

colormap (Autumn) 

hold on 

plot (xout ,w_exact, ’-k’, ’LineWidth’ ,3) 
xlabel(’\it m’,’FontSize’ ,25) 
ylabel(’Estimated \it P(m,100)’,’FontSize’ ,25) 


Figure 8.2.2 illustrates the results of simulating random walk. 
e Example 8.2.1: On the probability of striking a barrier 


An important question in engineering is what is the probability that a given random sys- 
tem will exceed its design constraints. Here we ask a similar question about one-dimensional 
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Figure 8.2.3: Several random walks from the origin to point (£, N). All of these walks would be excluded 
from our calculations because they either cross or touch the line m = € before the final step. 


Brownian motion: What is the probability that after taking N steps the particle arrives at 
€ without ever having touched or crossed the line m = € at any earlier step? We will do it 
exactly and then confirm our results using MATLAB. 

The arrival of the particle at € after N steps implies that its position after N — 1 steps 
must have been either € — 1 or € + 1. However, a trajectory from (+1, N — 1) to (£, N) is 
not allowed because it must have crossed the line m = € earlier. On the other hand, not all 
trajectories arriving at (£, N) from (€ — 1, N — 1) are acceptable because a certain number 
will have touched or crossed the line m = € earlier than its last step. See Figure 8.2.3. Thus 
the number of permitted ways of arriving at é for the first time after N steps equals all 
possible ways of arriving at € minus any arrivals from (€ — 1, N — 1) and any arrivals that 
crossed or touched the line m = € earlier than the N — 1. 

From our previous work, the number of possible ways from the origin to (€, N) is 


N! 


: (8.2.5) 
Fs Oy OVS 
The number of possible ways from the origin to (£ + 1, N — 1) is 
N-—1)! 
( (8.2.6) 


EDICE 
Finally, the number of trajectories arriving at (€ — 1, N — 1) but having an earlier contact 
with, or a crossing of, the line m = € is also 
(N — 1)! 
[N +]! [3(N — € = 2)]"" 


since it equals the number of trajectories that arrive at (€ + 1, N — 1). From Figure 8.2.3 
we see that, due to symmetry, the trajectory that leads to (£ + 1, N — 1) also leads to 


(8.2.7) 
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One of the great mathematicians of the twentieth century, Norbert Wiener (1894-1964) graduated 
from high school at the age of 11 and Tufts at 14. Obtaining a doctorate in mathematical logic 
at 18, he repeatedly traveled to Europe for further education. His work extends over an extremely 
wide range from stochastic processes to harmonic analysis to cybernetics. (Photo courtesy of the 
MIT Museum with permission.) 


(€—1, N—1). Consequently the number of trajectories from the origin to (£, N) that have 
never touched or crossed m = € is 


N! j (N — 1)! 
BANOE -9 ENHAN -E-D 


(8.2.8) 


or 
£ N! 


NEVO]! [GOV] 
The probability P(€, N) that we are seeking is 
_€ N! i 
PEN = Amoroa (3) . (8.2.10) 


For large N, P(€, N) is approximately given by 


P(E, N) = iel 5) (8.2.11) 


We can also compute this probability using the MATLAB code given above. In this 
code we replace the counting process location(icount) = m(N+1); by 


(8.2.9) 


b = sort (m); 

if ( (m(NN+1) == b(NN+1)) & (b(NN+1)>b(NN)) ) 
jcount = jcount + 1; 

location(jcount) = m(NN+1); 

end 


where we initialize jcount = 0 at the beginning. The idea behind this code is as follows: 
For each of the icount trajectories, we use the MATLAB routine sort to arrange them from 
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Figure 8.2.4: The probability P(€, N) that a particle will reach the point m = € without the particle ever 
crossing or touching the line m = € earlier than N = 100. The solid line is the theoretical probability given 
by Equation 8.2.11. Here 50,000 random walks were taken. 


the left-most to the right-most position. To be included in the count of particles reaching 
m = € at step N, the last position of the particle must be (£, N) and it may never have 
reached or crossed m = €. The if condition ensures that both conditions are met. If 
they are, that particular walk is accepted. Once again, the various right-most positions are 
binned and the probability is computed. Figure 8.2.4 illustrates this process using 5000 
random walks and this result is compared with the probability given by Equation 8.2.11. 


e Example 8.2.2: Wiener process 


Consider the time interval (0, ¢] and let us subdivide it into subintervals of length At 
so that there are t/At subintervals. Suppose now that a particle, initially at x = 0, takes a 
step (in one space dimension) at the times At, 2At,... and that the size of the step is either 
Ax or —Az, with a probability of 4 that the step is to the left or right. The position of 
the particle X(t) at time t is a random walk, which has executed t/At steps. Because the 
position depends on the choice of At and Az, X(t) depends upon t, At and Az. 
Mathematically we can describe this random process by 


t/At 
X(t) = 30 Z, (8.2.12) 
n=1 


where the Z;’s are independent and identically distributed with 


P(Z, = Az) = P(Z; = —Az) = §, (8.2.13) 


and n = 1,2,.... For each Z;, 


It6’s Stochastic Calculus 403 


Presently we have said nothing about the relationship between At and Az except 
that both are small. However, we cannot have just any relationship between them because 
the variance would be either zero or infinite. The only reasonable choice is Ax = VAt, 
which makes Var|X (¢)] = t for all values of At. In the limit At > 0 the random variable 
X(t) converges into a random variable, hereafter denoted by B(t), with the properties that 
E|B(t)| = 0 and Var[B(t)] = t. The collection of random variables {B(t),t > 0} is a 
continuous process in time and called a Wiener process. 


Our previous example shows that Brownian motion and the Wiener process are very 
closely linked. Because Brownian motion occurs in so many physical and biological pro- 
cesses, we shall focus on that motion (and the corresponding Wiener process) exclusively 
from now on. We define the standard Brownian motion (or Wiener process) B(t) as a 
stochastic process that has the following properties: 


1. It starts at zero: B(0) = 0. 

. Noting that B(t)— B(s) ~ N(0,t—s), E{[B(t)— B(s)|?} = t—s and Var{[B(t) — 
B(s)|?} = 2(t — s)?. Replacing t with t + dt and s with t, we find that 
E{{dB(t)|?} = dt. 

. It has stationary and independent increments. Stationary increments means 
that B(t + h) — B(n +h) = B(t) — B(n) for all h. An independent increment 
means B(t2) — B(ti),...,B(tn) — B(tn—1) are independent random variables. 

. Because increments of Brownian motion on adjacent intervals are independent 
regardless of the length of the interval, the derivative will oscillate wildly as 
Ax — 0 and never converge. Consequently, Brownian motion is nowhere differ- 
entiable. 

. It has continuous sample paths, i.e., “no jumps.” 


. The expectation values for the moments are given by 


(2n)!t” 
nian ? 


E(B?" (t)] = 


and E[B?"~1(t)] = 0, 


where n > 0. See Problem 1 at the end of Section 8.4. 


Problems 
1. Show that F{sin[aB(t)]} = 0, where a is a real. 


2. Show that 


Bfeos{aB(t))} = > SP" en”, 


where a is a real. 
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3. Show that E{exp[aB(t)]} = exp(a?t/2), where a is a real. 
Project: Probabilistic Solutions of Laplace’s Equation 


Laplace’s equation can be solved using finite difference or finite element methods, re- 
spectively. During the 1940s, the apparently unrelated fields of random processes and 
potential theory were shown to be in some sense mathematically equivalent.? As a result, it 
is possible to use Brownian motion to solve Laplace’s equation, as you will discover in this 
project. The present numerical method is useful for the following reasons: (1) the entire 
region need not be solved in order to determine potentials at relatively few points, (2) com- 
putation time is not lengthened by complex geometries, and (3) a probabilistic potential 
theory computation is more topologically efficient than matrix manipulations for problems 
in two and three spatial dimensions. 

To understand this technique,!? consider the following potential problem: 


oo + ot =O, O0<ax<l1, O0<y<l, (8.2.17) 
subject to the boundary conditions 
u(x, 0) =0, u(x, 1) = x, 0<a2<1, (8.2.18) 
and 
u(0, y) = u(1, y) = 0, 0<y<l. (8.2.19) 


If we introduce a uniform grid with Az = Ay = As, then the finite difference method yields 
the difference equation: 


4u(i,j) = uli +1,j)+uli— i, j) + uli, j +1) + ul, 7 — 1), (8.2.20) 


with i,j = 1, N — 1 and As = 1/N. 

Consider now a random process of the Markov type in which a large number Nj, of non- 
interacting particles are released at some point (x1, yi) and subsequently perform Brownian 
motion in steps of length As each unit of time. At some later time, when a few arrive at 
point (x,y), we define a probability P(i, j) of any of them reaching the boundary y = 1 
with potential uz at any subsequent time in the future. Whenever one of these particles 
does (later) arrive on y = 1, it is counted and removed from the system. Because P(i, j) is 
defined over an infinite time interval of the diffusion process, the probability of any parti- 
cles leaving (x,y) and arriving along some other boundary (where the potential equals 0) 
at some future time is 1 — P(i, j). Whenever a particle arrives along these boundaries it is 
also removed from the square. 

Having defined P(i, j) for an arbitrary (x, y), we now compute it in terms of the proba- 
bilities of the neighboring points. Because the process is Markovian, where a particle jumps 
from a point to a neighbor with no memory of the past, 


9 See Hersh, R., and R. J. Griego, 1969: Brownian motion and potential theory. Sci. Amer., 220, 
67-74. 


10 For the general case, see Bevensee, R. M., 1973: Probabilistic potential theory applied to electrical 
engineering problems. Proc. IEEE, 61, 423-437. 
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Figure 8.2.5: Four Brownian motions within a square domain with Ax = Ay. All of the random walks 
begin at grid point i = 4 and j = 6. 


where p(i + 1, jli, 7) is the conditional probability of jumping to (x + As, y), given that the 
particle is at (x,y). Equation 8.2.21 evaluates P(i,7) as the sum of the probabilities of 
reaching y = 1 at some future time by various routes through the four neighboring points 
around (x,y). The sum of all the p’s is exactly one because a particle at (x, y) must jump 
to a neighboring point during the next time interval. 

Let us now compare Equation 8.2.20 and Equation 8.2.21. The potential u(i, j) in 
Equation 8.2.20 and P(i, j) becomes an identity if we take the conditional probabilities as 


pli +1, jli j) = p- 1,514.9) = p j + 1i 3) = pij — 1i j) = E, 


and if we also force u(i, N) = P(i, N) = i, u(i,0) = P(i,0) = 0, u(0,7) = P(0,j) = 0, 
and u(N, j) = P(N, j) = 0. Both the potential u and the probability P become continuous 
functions in the space as As — 0, and both are well behaved as (x, y) approaches a boundary 
point. A particle starting along y = 1, where the potential is uz, has a probability uz of 
arriving there; a particle starting on the remaining boundaries, where the potential is zero, 
is immediately removed with no chance of arriving along y = 1. From these considerations, 
we have 


where N is the number of particles starting at (x, y) and N;, equals the number of particles 
that eventually—after infinite time—arrive along the entire boundary at potential ug. This 
sum includes the boundary y = 1 and (trivially) the remaining boundaries. 


Step 1: Develop a MATLAB code to perform two-dimensional Brownian motion. Let U 
be a uniformly distributed random variable lying between 0 and 1. You can use rand. If 
0<U< i, take one step to the right; ; <U< $, take one step to the left; if $ <U< 3, 
take one step downward; and if 3 < U < 1, take one step upward. For the arbitrary point 
i,j located on a grid of N x N points with 2 < i,j < N — 1, repeatedly take a random step 
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50 realizations 200 realizations 


800 realizations 3200 realizations 


Figure 8.2.6: Solution to Equation 8.2.17 through Equation 8.2.19 using the probabilistic solution method. 


until you reach one of the boundaries. Record the value of the potential at the boundary 
point. Let us call this result u-k (1). Figure 8.2.5 illustrates four of these two-dimensional 
Brownian motions. 


Step 2: Once you have confidence in your two-dimensional Brownian motion code, generalize 
it to solve Equation 8.2.17 through Equation 8.2.19 using runs realizations at some interior 
grid point. Then the solution u(i,j) is given by 


Step 8: Finally, plot your results. Figure 8.2.6 illustrates the potential field for different 
values of runs. What are the possible sources of error in using this method? 


8.3 ITO’S STOCHASTIC INTEGRAL 


In the previous section we noted that Brownian motion (the Wiener process) is nowhere 
differentiable. An obvious question is what is meant by the integral of a stochastic variable. 


Consider the interval [a, b], which we subdivide so that a = to < tı < t2 < -+ < tn =b. 
The earliest and simplest definition of the integral is 


f(t)dt = lim X. f(r) Ati, (8.3.1) 
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where ti—1 < T; < t; and At; = ti — t;_,. In the case of the classic integral, the integration 
is with regards to the increment dt. 

It’s integral is an integral where the infinitesimal increment involves Brownian motion 
dB(t), which is a random variable. Before we can define this integral, we must introduce 
two important concepts. The first one is nonanticipating processes: A process F(t) is a 
nonanticipating process if F(t) is independent of any future increment B(s) — B(t) for any 
s and t where s > t. Nonanticipating processes are important because It6’s integral applies 
only to them. 

The second important concept is convergence in the mean square sense. It is defined 
by 
2 


b 
lim E? |S, — f F(t)dB(t)| > =0, (8.3.2) 
noo a 
where Sn is the partial sum 
Sn = X F(t-1) [B(ts) — B(ti-1)]. (8.3.3) 


We are now ready to define the It6 integral: It is the limit of the partial sum Sn: 


b 

ms—lim Sn = / F(t) dB(t), (8.3.4) 
noo a 

where we denoted the limit in the mean square sense by ms—lim. Combining Equation 8.3.3 

and Equation 8.3.4 together, we find that 


n 


b 
J FEBO = fim D fa, BO) BE) - Ba], (8.3.5) 


where t; = iAt and At = (b — a)/N. As one might suspect, 


[ew = B(b) — B(a). (8.3.6) 


Because F(t) and dB(t) are random variables, so is Itô’s integral. 

The results from Equation 8.3.6 would be misunderstood if we think about them as we 
do in conventional calculus. We cannot evaluate the right side of Equation 8.3.6 by looking 
up B(t) in some book entitled “Tables of Brownian Motion.” This equation only holds true 
for a particular realization (sample path). 


e Example 8.3.1 


Let us use the definition of the It6 integral to evaluate It6 integral i B(x) dB(a). In 
the present case, 


Sn = D B(xi—1) |[B(z:) — B(zi—1)], (8.3.7) 
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2 


? 


where x; = it/n. Because 2a(b — a) = b? — a? — (b — a) 


Sn = ; È B?(z;) — ; B? (a1) — ; yy [B(a;) — Blei)? (8.3.8) 
= 1B) — 5 Ble) - Boa)? (8.3.9) 
Therefore, 
ms—lim S„ = 4.B?(t) — + ms—lim J [B(a;) — B(a;_1)]? (8.3.10) 
= 1B?) - : (8.3.11) 
As a consequence, 
| B(n) dB(n) = 3B°(t) — $, (8.3.12) 
or 
b b—a 
/ B(t) dB(t) = 4[B?(b) — B?(a)| a (8.3.13) 
Consider now the derivative of B?(t), 
d|B? (t)] = [B(t + dt) — B(t)|? = 2B(t) dB(t) + dB(t) dB(t). (8.3.14) 


In order for Equation 8.3.12 and Equation 8.3.14 to be consistent, we arrive at the very 
important result that 


[dB(t)]? = dt (8.3.15) 


in the mean square sense. We will repeatedly use this result in the remaining portions of 
the chapter. 


Because the Ito integral is a random variable, two important quantities are its mean 


and variance. Let us turn first to the computation of the expectation of he flt, BO] dB(t). 
From Equation 8.3.5 we find that 


ef [ s.20) aso) = fap» itn BOB} (8.3.16) 


= fn, 2 PeBe PABA = 0 (8.3.17) 


Therefore 


ef f rezo aso} =0. (8.3.18) 
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To compute the variance, we begin by noting that 


b 2 2 
tf fit, B(t)] ao) = ay B(ti—1)]A m (8.3.19) 
= lim Dri i-1, B(ti-1) (AB;)? (8.3.20) 


an >>> fti-1, B(ti-1)] AB: fltj-1, B(tj-1)JAB;. 


i=1 j=1 
ij 


Taking the expectation of both sides of Equation 8.3.20, we have that 


b 2 n 
Alf fe. BOLBE} | = 4a, D BUM Bl PEAR" (8.3.21) 
+ 2 lim >, Ds E{ f(ti-1, B(ti-1) |} E[ABi] E{ f(tj-1, B(tj-1)]}E[AB;] 
a 
= lim yest in BGG — ti). (8.3.22) 


At0 


The double summation vanishes because of the independence of Brownian motion. There- 
fore, the final result is 


Bf nso )| dB(t oy = f Eh, B(t)]} dt. (8.3.23) 


e Example 8.3.2 


Consider the random number X = f? vtsin[B(t)] dB(t). Let us find E(X) and E) 
From Equation 8.3.18, we have that E(X) = 0. For that reason, var(X) = E(X?) and 


Var(X) = E(X?) -f B{| ve sin[B( DP} at= f EISBN a (8.3.24) 


2 =] nny dt = / - i 3 oe] dt (8.3.25) 
© ¢_4)n+19n 
=- È CUE tl at = aa (O72 — a”t?). (8.3.26) 


The value of E{cos|2B(t)]} follows from Problem 2 at the end of the last section. 


410 Advanced Engineering Mathematics: A Second Course 


Table 8.3.1 gives a list of It6 stochastic integrals. Most of these results were not derived 
from the definition of the Ito stochastic integral but from It6 lemma, to which we now turn. 


Problems 


Consider the random variable X = if flt, B(t)] dB. Find E(X) and Var(X) for the follow- 
ing f[t, B(t)]: 


1. ft, BQ) =t 2. flt, B(t)] = tB(t) 
3. f lt, BE) = |BO| 4. FIF, B(t)| = vt exp[B(e)] 


5. If X = f? f(t){sin[B(t)] + cos[B(t)]} dB(t), show that var(X) = f? f?(t) dt, if f(t) is a 
real function. 


Project: Numerical Integration of It6’s Integral 


Equation 8.3.5 is useful for numerically integrating the Ito integral 


I flz, B(x)]|dB(x). 


Write a MATLAB script to check Example 8.3.1 for various values of n when t = 1. How 
does the error vary with n? 


Project: Numerical Check of Equations 8.3.18 and 8.3.23 


Using the script from the previous project, develop MATLAB code to compute Equation 
8.3.18 and Equation 8.3.23. Using a million realizations (sample paths), compare your 
numerical results with the exact answer when a = 1, b = 1, and f[t, B(t)] = vt sin[B(¢)]. 


8.4 1TO’S LEMMA 


Before we can solve stochastic differential equations, we must derive a key result in 
stochastic calculus: Itô’s formula or lemma. This is stochastic calculus’s version of the 
chain rule. 

Consider a function f(t) that is twice differentiable. Using Taylor’s expansion, 


df (B) = f(B + dB) — f(B) = f'(B) dB +43 f" (B) (dB)? +-->, (8.4.1) 


where B(t) denotes Brownian motion. Integrating Equation 8.4.1 from s to t, we find that 


| df(B) = f[BQ] - f{B(s)] = f f'(B)dB+: 1 f"(B)dz+:, (8.4.2) 


S 


because [dB(x)]? = dx. The first integral on the right side of Equation 8.4.2 is an Itô’s 
stochastic integral while the second one can be interpreted as the Riemann integral of f” (B). 
Therefore, It6’s lemma or formula is 


FIBON- f1B(6)) = f Baw +3 f 7(B) de (8.4.3) 


It6’s Stochastic Calculus 411 


Table 8.3.1: A Table of Itô Stochastic Integrals with t > 0 andb>a>0 


1. 
[as = B(b) — Ba) 
Ds : 
| B(n) AB (n) = $B — 0 
3. : 
f [B?(n) — n] dB(n) = 1B? (£) — tB() 
4. : ? 
Í ndB(n) = tB(t) - p B(n) dy 
0 0 
5. ; : 
f B? (n) dB(n) = 1 B? (t) — f B(n) dn 
6. F 
i P dB(n) = xen? sin[AB(t)] 
0 
Ls : i 
f e^®n/? sinf[AB(n)] dB(n) = 5 {1 — e%t/2 cos[\B(t)]} 
8. : i 
f exp[-4\n + AB()] dB() = +5 {exp[- 4At + AB()] — 1} 
á é B?(n)] dB(n) B?(b) ) 
n n —1/2 —1/2 a 
[ ro] F pr b exp] |- exp] 
10. i i 3 
/ f(n) dB(n) = FEBE) - 1 {'(n) Bn) dn 
11. 
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for t > s. 
e Example 8.4.1 


Consider the case when f(t) = t? and s = 0. Then, It6’s formula yields 


B?(t) — B?(0 = 2 f B B(x) dB(x j- fae (8.4.4) 


Evaluating the second integral and noting that B(0) = 0, we again obtain Equation 8.3.12, 
that 


f B(x) dB(x) = 4[B?(t) — t]. (8.4.5) 


e Example 8.4.2 
Consider the case when f(t) = e% and s = 0. Then, Itô’s formula yields 
t a2 t 
eB) 1 = af eB) dB(x) + sf eB) dr. (8.4.6) 
0 0 


Computing the expectation of both sides, 


Ble a2) a -$ fale (8.4.7) 


Solving this integral equation, we find that E[e??] = e@ t/2, a result that we found earlier 
in Problem 2, Section 8.2. 


e Example 8.4.3 


If f(t) = sin(At), A > 0, then Itd’s formula gives 
t 
sin[A B(t) E cos[AB(n)] dB(n) — nef sin[AB(n)]| dn. (8.4.8) 
0 


Taking the expectation of both sides of Equation 8.4.8, we find that 
t 
E{sin[AB(t)|} = -3x f E{sin[AB(n)]} dn. (8.4.9) 
0 
Setting g(t) = E{sin[A B(t)]}, then 


g(t) = -x | g(n) dn. (8.4.10) 


The solution to this integral equation is g(t) = 0. Therefore, E{sin[AB(t)]} = 0. 
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Educated at the Imperial University of Tokyo, Kiyoshi It6 (1915-2008) applied the techniques of 
differential and integral to stochastic processes. Much of It6’s original work from 1938 to 1945 was 
done while he worked for the Japanese National Statistical Bureau. After receiving his doctorate, 
It6 became a professor at the University of Kyoto from 1952 to 1979. (Author: Konrad Jacobs, 
Source: Archives of the Mathematisches Forschungsinstitut Oberwolfach.) 


The second version of It6’s lemma begins with the second-order Taylor expansion of 
the function f(t, x): 


Flt + dt, B(t + dt)| — flt, BŒ] = Filt, BO] dt + felt, B) dB) 
+ H fult, BŒ] (dt)? + felt, BD] dtdB(t) (8.4.11) 
T feat, B(t)] [dB(t)]?} aire 


Here we assume that f[t, B(t)] has continuous partial derivatives of at least second order. 
Neglecting higher-order terms in Equation 8.4.11, which include the terms with factors such 
as (dt)? and dtdB(t) but not [dB(t)|? because [dB(t)|? = dt, our second version of Itô’s 
lemma is 


flt, B(t)] — fis, B(s)] a {filn, B(n)] + $ fez, B(n)]} ay + f faln, B(n)] dB(n) 
° i (8.4.12) 
if t >s. 


414 Advanced Engineering Mathematics: A Second Course 
e Example 8.4.4 
Consider the function f(t, £) = e*~'/?. Then, 
filt, £) = — 4e? folt, x) =e", and fea(t,x) = et. (8.4.13) 


Therefore, from Itô’s lemma, we have that 


t 
eP(t)—-t/2 = eB(s)—s/2 — I e7/2eB™ dB(n). (8.4.14) 


S 


e Example 8.4.5: Integration by parts 
Consider the case when F'(t,x) = f(t)g(x). The Itô formula gives 
dif Oga) = {F OIBO] + FOBO) at + fg [BO] BO). (8.4.15) 


Integrating both sides of Equation 8.4.15, we find that 


b b b b 
I f(g [BO] AB(t) = FOBO - j f'(HglBW] at — } j f(t)g"[B()] dt, (8.4.16) 


which is the stochastic version of integration by parts. 
For example, let us choose f(t) = e® and g(x) = sin(x). Equation 8.4.16 yields 


t t 


| e°" cos[B(n)] dB(n) = e®" sin[B(n)]| — af e*” sin[B(n)] dn + af e°” sin[B(n)] dn 
0 0 0 


0 
(8.4.17) 
t 
= e™ sin[B(t)] — (a — 3) J e™ sin[ B(n)] dn. (8.4.18) 
0 
In the special case of a = F, Equation 8.4.18 simplifies to 
t 
Í e™" cos[B(n)| dB(n) = et/? sin|B(t)]. (8.4.19) 
0 


An important extension of Itô’s lemma involves the function f[t, X(t)] where X(t) is no 
longer simply Brownian motion but is given by the first-order stochastic differential equation 


dX(t) = cX(t) dt +o X(t) dB(t), (8.4.20) 
where c and o are real. The second-order Taylor expansion of the function f[t, X (t)] becomes 


flt + dt, X(t + dt)] — fit, XO] = Flt, XH] dt + falt, X] dX (t) (8.4.21) 
+ 5{feelt, XO] (dt)? + felt, X (€)] dt dX (t) + foalt, XO XOP} +--- 
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Next, we substitute for dX(t) using Equation 8.4.20, neglect terms involving (dt)? and 
dt dB(t), and substitute [dB(t)]? = dt. Consequently, 


df = fit + dt, X(t + dt)] — Flt, X(0)] (8.4.22) 
= o X(t) falt, X(t)] dB(t) + fs lt, X (€)] + cX (t) felt, XO] + 40X? (t) feelt, xol} dt. 
(8.4.23) 


The present extension of Itô’s lemma reads 


fit, X(0)] = fX = f {fim xo + eX(n)feln, X(n)] + 402X?(n) fas in. xt) dn 


$ / oX(n)Feln, X(m)] €B(n) (8.4.24) 
= {fel XC + 19?X?(n) foo nx) dn 
de / feln, X(n)] 4X (n), (8.4.25) 
where 
dX (n) = cX (n) dn + oX (n) dB(n) (8.4.26) 
and t > s. 


We can finally generalize Itô’s formula to the case of several Itô processes with respect 
to the same Brownian motion. For example, let X(t) and Y(t) denote two Itô processes 
governed by 

dX(t) = AW) (t) dt + ACD (t) dB(t), (8.4.27) 


and 


dY (t) = AT? (t) dt + AC? (t) dB(t). (8.4.28) 
For stochastic process f|t, X(t), Y(t)], the Taylor expansion is 
df lt, XOY O] = Felt, X(t), Y H] dt + felt, X(t), Y ()] dX (t) 
+ fylt, X(t), Y(t) dY (t) (8.4.29) 
+ $fralt, X(t), ¥ (JAP (ACY ©) dt + 3 faylt, XH, YHA? QAP (t) at 
+ 4 fyalt, X(t), Y(QAP? QAP (t) dt + E fyylt, XE), Y QAP? (AP) (t) dt. 


e Example 8.4.6: Product rule 


Consider the special case f(t,z,y) = xy. Then ft =0, fe = yY, fy = £, fea = fyy = 9, 
and fry = fyz = 1. In this case, Equation 8.4.29 simplifies to 


d[X (t)Y (t)] = Y(t) dX (t) + X(t) dY(t) + ACP lt, X(t), YAP |t, X(t), Y (b)] dt. 
(8.4.30) 
A very important case occurs when ACD ft, X(t), Y(t)] = 0 and X(t) = g(t) is purely 
deterministic. In this case, 


dig(t)Y (t)] = Y (t) dg(t) + g(t) dY (2). (8.4.31) 
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This is exactly the product rule from calculus. 
Problems 


1. (a) Use Equation 8.4.3 and f(t) = t” to show that 
t —1 t 
B” (t) = n f B”! (2) dB(x) + n | B”? (x) dx. 
0 0 


(b) Show that 
E(B" (t)] = noe) f E|B"-?( 


(c) Because E[B(t)] = 0 and E[B?(t)] = t, show that 


(2k)! y 


B[B*+1(4)] =0, and E[B**(t)] ~ ORI 


2. Let f(t, x) = 27t and use Itô’s formula to show that 


J Beare 7B(n) dB(n) = tB? (t) — t?/2. 
0 0 


3. Let f(t, x) = x°/? and use It’s formula to show that 


[ Bec anon = 3070-3 [Ba 


4. Let f(t, £) = 23/3 — tx and use Itô’s formula to show that 


T [B?(n) — n] dB(n) = }B°(t) - t B(). 


5. If f(x) is any continuously differentiable function, use Equation 8.4.29 to show that 


f f dB(n) = f(t)B@) - Í f'()B 


6. If f(t) =e’, use the previous problem to show that 


f a dB(n) =  B(t) — i e" B(n) dn. 


7. Let G(x) denote the antiderivative of g(a). Use Equation 8.4.3 to show that 


b 


b 
1 a(B(t)] dB(t) = GIB) 


a 
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8. (a) If g(x) = ze”, use Problem 7 to show that 
t 
B(n)e®™ dB(n) = [B(t) — 1]e”® +1- af [B(n) + eB dn. 
0 
(b) Use Equation 8.3.18 to show that 
t 
E Bite? = Ele O| —1+ af E B(n)e2™ + EleP™|\ dn 
Poet jsa p laa eee eee) 
t 
=e/?_14 >f fen? + E|Bme®]} dn. 
0 


(c) Setting g(t) = E[B(t)e?], use Laplace transforms to show that 


E| BEP] = te’, 
9. (a) If g(x) = 1/(1 + xz?), use Problem 7 to show that 


EG) ss _ fi B) 
fp melt | Ea 


(b) Use Equation 8.3.18 to show that 


á B(n) Sipe: 
fe} dn = —Efarctan|B(t)]}. 


(c) Because 


BUS ot V8. a LS BVS Oe, 
16 (1 + 22)? 16’ 16 o [1+ B2(n)|? 16 
show that 
3v3 3v3 


ETA < E{arctan|B(t)]} < ET 


10. If g(x) = x/(1 +27), use Problem 7 to show that 


t B(n) eerie ı [Ý 1- B?(n) 
J p Bn +O- r 


11. Use integration by parts with f(t) = eft and g(x) = — cos(x) to show that 


J ef" sin[B(n)] dB(n) = 1 — e% cos|B(t)] + (8-5) i e?" cos|B(n)] dn. 
0 0 
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Then, take 6 = 4 and show that 


i e"/? sin{B(n)| dB(n) = 1 — et”? cos[B(t)]. 


12. Redo Example 8.4.3 and show that E{cos[\B(t)]} = e7>*/2, > 0. 


13. Use trigonometric double angle formulas to show that 


(a) E{sin[t + AB(t)]}} =e */? sin(t), 
and 

(b) E{cos|t + AB(t)]} =e */? cos(t), 
when À > 0. 


14. Following Example 8.4.4 with f(t, x) = dexp( AL — à?t/2), A > 0, show that 


[ exp [tago = =] dB(n) = + {exp jaso 7 a] z i} . 


Le) cos[AB(n)] dB (n) = te(22) sin[\B(t)]. 


16. Following Example 8.4.4 with f(t, x) = — exp(A?t/2) cos(Ax), A > 0, show that 


[ ep (22) sin[AB(n)] dB(n) = k fı 7 ep( 2) cosAB()I} l 


17. Following Example 8.4.4 with f(t, x) = t1? exp[z?/(2t)], show that 


[eye] 20] 20 oop] BO] — Pap AO] 


2t Br? 
18. The average of geometric Brownian motion on [0, t] is defined by 


1 t 
G(t) = F eB) dn. 
t Jo 


Use the product rule to find dG(t). Hint: Take the time derivative of tG (t) = le eB) dn. 
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8.5 STOCHASTIC DIFFERENTIAL EQUATIONS 


We have reached the point where we can examine stochastic differential equations. Of 
all the possible stochastic differential equations, we will focus on Langevin’s equation!!—a 
model of the velocity of Brownian particles. We will employ this model in a manner similar 
to that played by simple harmonic motion in the study of ordinary differential equations. 
It illustrates many of the aspects of stochastic differential equations without being overly 
complicated. 


e Example 8.5.1 


Before we consider the general stochastic differential equation, consider the following 
cases where we can make clever use of the product rule. For example, let us solve 


dX(t) = [t + B?(t)| dt + 2tB(t) dB(t), X(0) = Xo. (8.5.1) 
In the present case, we can find the solution by noting that 
dX(t) = B?(t) dt + t[2B(t) dB(t) + dt] = B?(t) dt + td[B?(t)] = d[tB? (0). (8.5.2) 
Integrating both sides of Equation 8.5.2, we find that the solution to Equation 8.5.1 is 
X(t) = tB? (t) + Xo. (8.5.3) 
Similarly, let us solve the stochastic differential equation 


b- X(t 
dX(t) = P= X10 dt + dB(t), 0<t<l, (8.5.4) 
with X(0) = Xo. 
We begin by writing Equation 8.5.4 as 


db- X(t)] b-— X(t) dB(t) 
dt = i 8.5. 
1-t (1 -t)? 1-t (8:55) 
Running the product rule backwards, 
b- X(t) dB(t) 
d = F 5. 
ee 856) 
Integrating both sides of Equation 8.5.6 from 0 to t, we find that 
b- X(t ‘dB 
W 4 x@) | CBO, (8.5.7) 
Solving for X(t), we obtain the final result that 
‘dB 
X(t) =b— p- x00 -9+0-9 f pe, (8.5.8) 
ee odes 


11 Langevin, P., 1908: Sur la théorie du mouvement brownien. C. R. Acad. Sci. Paris, 146, 530-530. 
English translation: Langevin, P., 1997: On the theory of Brownian motion. Am. J. Phys., 65, 1079-1081. 
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In the present case we cannot simplify the integral in Equation 8.5.8 and must apply nu- 
merical quadrature if we wish to have numerical values. 


In the introduction we showed that the solution to Langevin’s equation: 
dX(t) = cX(t)dt+ odB(t), X(0) = Xo, (8.5.9) 
is 
t t 
X(t) = Xo + a X(n) dn + o f dB(n). (8.5.10) 
0 0 
An obvious difficulty in understanding this solution is the presence of X(s) in the first 
integral on the right side of Equation 8.5.21. 


Let us approach its solution by considering the function f (t,x) = e~“x. Then, by Itô’s 
lemma, Equation 8.4.16, 


flt, X()] — X(0) =| {feln, X] + eX (m) fein, XO] + 507 feam X(n)]} dn 


$ | o faln, X(n)] aB(n), (8.5.11) 
0 


because f[0,X(0)] = X(0). Direct substitution of f(t, x) into Equation 8.5.11 yields 
t 
e ¢X(t)-— Xo = o f e “ dB(n). (8.5.12) 
0 
Finally, solving for X(t), we obtain 
t 
X(t) = X(O)e + ae f e “"dB(n), (8.5.13) 
0 


an explicit expression for X(t). For the special case when Xo is constant, X(t) is known as 
an Ornstein-Uhlenbeck process.!? 
An alternative derivation begins by multiplying Equation 8.5.9 by the integrating factor 
e~* so that the equation now reads 
e “dX (t) — ce~“ X (t) dt = oe“ dB(t). (8.5.14) 
Running the product rule, Equation 8.4.23, backwards, we have that 
dle“ X(t)] = ce“ dB(t). (8.5.15) 
Integrating both sides of Equation 8.5.15, we obtain Equation 8.5.12. 


e Example 8.5.2: Exact stochastic differential equation 


Consider the stochastic differential equation 


X(t) =x +e f X(s) ato f X(s) dB(s), (8.5.16) 


12 Uhlenbeck and Ornstein, op. cit. 
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Figure 8.5.1: Ten realizations (sample paths) of geometric Brownian motion when c = 0.1, ø = 0.5, and 


X(0) = 1. The heavy line is the mean of X (t). 


with c,o > 0. 
If X(t) = f[t, B(t)], then by Itd’s lemma, Equation 8.4.9, 


t t 
X(t) =x0)+ | { fils, B(s)] + > feels, B(s)]} as+ f fxs, B(s)] dB(s). 
0 0 
Comparing Equation 8.5.16 and Equation 8.5.17, we find that 


cf (t,x) = filt, x£) + faa (t, £), 


and 
of(t,x) = fr (t,x). 
From Equation 8.5.19, 
fralt, £) = 0 falt, £) = 0° f (t, £). 


Therefore, Equation 8.5.18 can be replaced by 


(c— 50°) f(t, x) = filt, a). 


(8.5.17) 


(8.5.18) 


(8.5.19) 


(8.5.20) 


(8.5.21) 


Equation 8.5.19 and Equation 8.5.21 can be solved using separation of variables, which 


yields 
f(t, £) = f(0,0)exp[ (c — 40°) t + ox], 


or 
X(t) = flt, B)] = X(0) exp[(c— $07) t+ o B(t)] . 


(8.5.22) 


(8.5.23) 


Thus, a stochastic differential equation can sometimes be solved as the solution of a deter- 
ministic partial differential equation. In the present case, this solution is called geometric 


Brownian motion. For its solution numerically, see Example 8.6.1. See Figure 8.5.1. 
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e Example 8.5.3: Homogeneous linear equation 
Consider the homogeneous linear stochastic differential equation 
dX(t) = c(t) X(t) dt + 01 (t)X (t) dB(t). (8.5.24) 
Let us introduce f(t, x)= ln(x). Then by It6’s lemma, Equation 8.4.21, 
df = d{ln(X)] = [e1(t) — 402 (t)] dt + o1(t) dB(t), (8.5.25) 


because f; = 0, fr = 1/x and fre = —1/x?. Integrating both sides of Equation 8.5.25 and 
exponentiating the resulting expression, we obtain 


X() =X(exn| [ teen = o a+ f a(n) aay). (8.5.26) 


e Example 8.5.4: General case 
Consider the homogeneous linear stochastic differential equation 
dX(t) = [a (t) X(t) + c2(t)] dt + [oi (t) X(t) + o2(t)] dB(t). (8.5.27) 
Our analysis begins by considering the homogeneous linear stochastic differential equa- 


tion 
í dY (t) = a (t)Y (t) dt + oi (t)Y (t) dB(t), Y(0) =1. (8.5.28) 


From the previous example, 


Y(t) = exp f T — Bo2(n)] an + f an aB(n)} . (8.5.29) 


Next, let us introduce two random variables, X; = 1/Y and Xə = X. Using Itô lemma 
f(t,x) = 1/x, then 


dXı = df(t, Y) = (=) = [of (t) — ci(t)] “ - Te (8.5.30) 
= [of (t) — c1 (t)] X1 (t) dt — o1 (t)Xı (t) dB(t), (8.5.31) 


since f, = 0, fe = —1/x? and fry = 2/2°. 
Using Equation 8.4.30, where X; is governed by Equation 8.5.31 and Xə is governed 
by 8.5.27 because Xə = X, 


d(Xı X2) = [c2 (t) — 01 (t)o2(t)] Xi (t) dt + o2(t) X41 (t) dB(t). (8.5.32) 
Upon integrating both sides of Equation 8.5.32, we have 
LF eis entra faye 
XXa- X0) = | lest) - otma] ey + fm Fey (85.33) 
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Consequently, our final result is 


xy =o {x00 + f em-an SE + footy BR), 5.94) 
0 n) 0 Y(n) 
where Y (t) is given by Equation 8.5.29. 
e Example 8.5.5: Stochastic Verhulst equation 
The stochastic Verhulst equation is 
dX (t) = aX(t)[M — X(t)] dt +bX(t)dB(t), X(0) = Xo. (8.5.35) 


We begin its solution by introducing ®(t) = 1/X(t). Then by Itô’s lemma, Equation 
8.4.21 with f(a) = 1/2, 


d®(t) = —®(t)[(aM — b?) dt + bdB(t)] + a dt, (0) =1/Xo. (8.5.36) 
To solve Equation 8.5.36, we use the results from Example 8.5.4 with c1 (t) = b? — aM, 


c2(t) = a, o1(t) = —b, and o2(t) = 0. Denoting e(t) = (aM — b?/2)t + bB(t), we can write 
Equation 8.5.34 as 


t 
&(t)e§ — (0) = af e€ dn, (8.5.37) 
0 
= E) t 
e 1 
2 ->= E) q 8.5.38 
a e š 0. 
X Xo I í (3248) 


Solving for X(t), we obtain the final result that 


= Xo exp[é(t)] l 
1+ aXo fa expl€(n)] dn 


(8.5.39) 


Problems 
1. Solve the stochastic differential equation 
dX(t) = de'/? B(t) dt +e? dB(t),  X(0) = Xo, 
by running the product rule backwards. 
2. Solve the stochastic differential equation 
dX(t) = e” [1 + 2B°(t)] dt + 2e” B(t)dB(t),  X(0) = Xo, 


by running the product rule backwards. Hint: Rewrite the differential equation dX(t) = 
e°”t[2B(t)dB(t) + dt] + (2e dt) B? (t). 


3. Solve the stochastic differential equation 


dX(t) = [1 + B(t)] dt + [t + 2B(t)] dB(t), X (0) = Xo, 
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by running the product rule backwards. Hint: Rewrite the differential equation dX(t) = 
2B(t)dB(t) + dt + B(t)dt+tdB(t). 


4. Solve the stochastic differential equation 
dX(t) = [3t? + B(t)|dt+tdB(t),  X(0) = Xo, 


by running the product rule backwards. Hint: Rewrite the differential equation dX(t) = 
3t? dt + [B(t) dt +tdB(t)]. 


5. Solve the stochastic differential equation 
dX(t) = B?(t) dt +2tB(t)dB(t), X(0) = Xo, 


by running the product rule backwards. Hint: Rewrite the differential equation dX(t) = 
t[2B(t) dB(t) + dt] + B?(t) dt — t dt. 


6. Find the integrating factor and solution to the stochastic differential equation 
dX(t) = [8 — aX (t)] dt + o0dB(t), X(0) = Xo, 

where B(t) is Brownian motion and a, 8 and o are constants. 

7. Find the integrating factor and solution to the stochastic differential equation 
dX(t) = [1 + 2X(t)] dt + e% dB(t), X(0) = Xo, 

where B(t) is Brownian motion. 


8. Find the integrating factor and solution to the stochastic differential equation 


dQ(t) 4 Q(t) 


oa dt = a dt 4 a) dB(t), QO) = Qo, 

where R and C are real, positive constants, and B(t) is Brownian motion. 

9. Find the integrating factor and solution to the stochastic differential equation!’ 
dX(t) = tX (t) dt +e™* dt +4B(t), te [0,1], 

with X(0) = Xo, and B(t) is Brownian motion. 

10. Find the integration factor and solution to the stochastic differential equation 
dX(t) = [4X (t) — 1] dt + 2dB(t), X(0) = Xo, 


where B(t) is Brownian motion. 


13 Khodabin, M., and M. Rostami, 2015: Mean square numerical solution of stochastic differential equa- 
tions by fourth order Runge-Kutta method and its applications in the electric circuits with noise. Adv. 
Diff. Eq., 2015, 62. 
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11. Find the integration factor and solution to the stochastic differential equation 
dX(t) = [2 — X (t)] dt + e* B(t) dB(t), X(0) = Xo, 
where B(t) is Brownian motion. 
12. Find the integration factor and solution to the stochastic differential equation 
dX(t) = [1 + X(t)] dt + e* B(t) dB(t), X(0) = Xo, 
where B(t) is Brownian motion. 
13. Find the integration factor and solution to the stochastic differential equation 
dX(t) = [4X (t) +1] dt + e’ cos[B(t)] dB(t), X(0) = Xo, 
where B(t) is Brownian motion. 
14. Find the integration factor and solution to the stochastic differential equation 
dX(t) = [t+ 4X (t)| dt +e! sin[B(t)|dB(t),  X(0) = Xo, 
where B(t) is Brownian motion. 


15. Following Example 8.5.2, solve the exact stochastic differential equation: 


dX(t) = e' [1+ B?(t)] dt + [1 + 2e'B(t)] dB(t), X(0) = Xo. 


Step 1: Show that f;+4fre =e'(1+27), and fr = 1+ 2er. 
Step 2: Show that f(t, x) = x + etr? + g(t). 
Step 3: Show that g(t) = Xo and X(t) = B(t) +e’ B? (t) + Xo. 


16. Following Example 8.5.2, solve the exact stochastic differential equation: 


dX (t) = {2¢B?(t) + 3¢ [1+ B(t)]} dt + [1 + 3t? B? (t)] dB, X(0) = Xo. 


Step 1: Show that fi + fes = 2t? + 3t? (1 + x), and fr = 3t?2? +1. 
Step 2: Show that f(t, £) = t?a3 +x + g(t). 

Step 3: Show that g'(t) = 3¢?. 

Step 4: Show that X(t) = t?[B3(t) + t] + B) + Xo. 


Using Equation 8.5.26, solve the following stochastic differential equations: 


17. dX(t) = t?X(t) dt + tX (t)dB(t), X(0) = Xo 
18. dX(t) = cos(t)X (t) dt + sin(t)X(t)dB(t),  X(0) = Xo 
19. dX(t) =In(t + 1)X(t) dt + Vnt +1) X(t)dB(t),  X(0) = Xo 
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20. dX(t) = In(t + 1)X(t) dt + tX(t) dB(t), X(0) = Xo 
21. Following Example 8.5.5, solve the stochastic differential equation 

dX (t) = [a X" (t) + bX (t)] dt + cX(t) dB(t), X(0) = Xo, 


where n > 1. 


Step 1: Setting P(t) = X'~"(t), use Itô’s lemma Equation 8.4.21 with f(z) = 1/x"~! to 
show that 
d®(t) = (1— n)®(t) [(b— in?) dt + cdB(t)] + (1—n)adt. 


Step 2: Setting ci(t) = (1 — n)b— n(1 — n)c?/2, c(t) = (1 — n)a, o1(t) = (1 — n)c, and 
o2(t) = 0, show that 


expl(n— DEO 1 t 
Rg ga lna f epla- Neto) dn 
` (n-d 1 ! 
exp|(n — t l 
a a e — Dela) dn 


where €(t) = (b — c?/2)t + cB(t). 
22. Following Example 8.5.5, solve the stochastic Ginzburg-Landau equation: 


dX(t) = [aex +2] dt+oadB(t),  X(0) = Xo. 


Step 1: Setting P(t) = exp[—cX (t)], use Itô’s lemma Equation 8.4.21 with f(x) =e to 
show that 
d®(t) = — (be — 40°) D(t) dt — oc (t) dB(t) — ac dt. 


Step 2: Setting cı (t) = 07c?/2 — be, co(t) = —ac, o1(t) = —oc, and o2(t) = 0, show that 


X(t) = Xo + bt + oB(t) *in{ ae f exp[cXo + bc + oc B(E)] ach. 


23. Following Example 8.5.5, solve the stochastic differential equation: 


dX(t) = {[1 + X(t)][1 + X7(t)]] dt + [1+ X?) dB(t), =X (0) = Xo. 


Step 1: Setting ®(t) = tan~![X(t)], use Itô’s lemma Equation 8.4.21 with f(x) = tan~!(a) 
to show that d®(t) = dt + dB(t). 


Step 2: Solving the stochastic differential equation in Step 1, show that 


X(t) = tan[tan™ (Xo) + t + B(d)). 
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8.6 NUMERICAL SOLUTION OF STOCHASTIC DIFFERENTIAL EQUATIONS 


In this section we construct numerical schemes for integrating the stochastic differential 
equation 
dX(t) = aX (t), t] dt + b| X (t), t] dB(t) (8.6.1) 


on to < t < T with the initial-value X (to) = Xo. 

Our derivation begins by introducing the grid to < tı < tg <- <tn <: <ty =T. 
For simplicity we assume that all of the time increments are the same and equal to 0 < 
At < 1 although our results can be easily generalized when this is not true. Now 


r EES E I aX (n),n] dn + f OLX (n), n] dB(n). (8.6.2) 


n n 


The crudest approximation to the integrals in Equation 8.6.2 is 


I omire at A (8.6.3) 


n 


tn+1 
J AX). n a(n) = UX (tn) sto] Br. (8.6.4) 
tn 

Substituting these approximations into Equation 8.6.2 yields the Euler-Marugama approx- 
imation.'* For the Itô process X(t) = {X(t), to < t < T}: 


Xanti = Xn + a(tn, Xn) (tn+1 = tn) + B(tn, Xn) (Bina ma B:,.) (8.6.5) 


for n = 0,1,2,...,N — 1 with the initial value Xo. 

When 6 = 0, the stochastic iterative scheme reduces to the conventional Euler scheme 
for ordinary differential equations. When b Æ 0, we have an extra term generated by the 
random increment AB, = B(tn41) — B(tn) where n = 0,1,2,...,N — 1 for Brownian 
motion (the Wiener process) B(t) = B(t),t > 0. Because these increments are independent 
Gaussian random variables, the mean equals E(AB,,) = 0 while the variance is E[(AB,,)?] = 
At. We can generate AB,, using the MATLAB function randn. 

An important consideration in the use of any numerical scheme is the rate of conver- 
gence. During the numerical simulation of a realization, at time t there will be a difference 
between the exact solution X(t) and the numerical approximation Y(t). This difference 
e(t) = X(t) —Y(t) will also be a random variable. A stochastic differential equation scheme 
converges strongly with order m, if for any time t, E(|e(t)|) = O[(At)”™] for sufficiently small 
time step At. The strong order for the Euler-Marugama method can be proven to be 5. 

To construct a strong order 1 approximation to Equation 8.6.1, we return to Equation 
8.6.2. Using Equation 8.4.12, we have 


taji n n 
Xn+1 — Xn = J fenon +f (aaz + ibaze) dé +f bax aB) dn 


tn 


+f ii faxat. f (ab, + 4b°bss) a+ f bb, aB(| dn (8.6.6) 


= a[X (tn), tn] At + b[X (tn), tn) ABn + Rn, (8.6.7) 


14 Maruyama, G., 1955: Continuous Markov processes and stochastic equations. Rend. Circ. Math. 
Palermo, Ser. 2,, 4, 48-90. 
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where 


tn+1 n 
Rn = J if bbz B®) dB(n) + higher-order terms. (8.6.8) 
t t 


n n 


Dropping the higher-order terms, 
tn+1 n 
Ra = BX En) tabe ta) ta | [S aB] aso. (8.6.9) 
t t 


Consider now the double integrals 


(AB,)? = ( ie Btn) ( I m dB(n)) = ie | w aB) dB(n). (8.6.10) 


n 


Now, 
(Uf am a= f° [am a f° [fam a 
+ [wom (8.6.11) 
=2f "| fase] a+ fase? (8612) 
=2f | faa] apy +a (8.6.13) 
ee 
i AB? = f aoe (8.6.14) 


n n 


Combining Equation 8.6.9, Equation 8.6.10, and Equation 8.6.13 yields 
Rn & OLX (tn), tr]ba[X (tn), tn] [((ABn)? — At] . (8.6.15) 


Finally, substituting Equation 8.6.15 into Equation 8.6.7 gives the final result, the Milstein 
method:!° 


O0(Xn, tn) 
o 


Xn41 = Xn +a(Xn, tn) Atn +b(Xn, tn) ABn + §0(Xn, tn) [(ABn} — At] . 


(8.6.16) 
e Example 8.6.1 


Consider the It6 process X(t) defined by the linear stochastic differential equation 


dX(t) = aX (t) dt + bX (t) dB(t), (8.6.17) 


15 


Milstein, G., 1974: Approximate integration of stochastic differential equations. Theory Prob. Applic., 
19, 557—562. 
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Figure 8.6.1: The numerical solution of the stochastic differential equation, Equation 8.6.17, using the 
Euler-Marugama (crosses) and the Milstein (circles) methods for various time steps h. The dashed line 
gives the exact solution. 


for t € [0,7]. If this It6 process has the drift a(x,t) = ax and the diffusion coefficient 
b(x,t) = bax, the exact solution (see Equation 8.5.16) is 


X(t) = Xo exp|(a = =) t+ 15(0| (8.6.18) 


for t € [0,7]. Figure 8.6.1 compares the numerical solution of this stochastic differential 
equation using the Euler-Marugama and Milstein method against the exact solution. Note 
that each frame has a different solution because the Brownian forcing changes with each 
realization. 


Although a plot of various realizations can give an idea of how the stochastic processes 
affect the solution, two more useful parameters are the sample mean and standard deviation 
at time tn: 


X(t) = = Xiltn) (8.6.19) 
and ; 
0 (ta) = = Te- r. (8.6.20) 


where J are the number of realizations and X; (tn) is the value of the random variable at 
time tn of the jth realization. 

In many physical problems, “noise” is the origin of the stochastic process and we suspect 
that we have a normal distribution N (u, o?) where u and o are the population mean and 
standard deviation, respectively. Then, using the sample statistics, Equations 8.6.20 and 
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2.5 3 3.5 4 


2 
Rt/L 


Figure 8.6.2: Eleven realizations as a function of the nondimensional time Rt/L of the numerical solution 
of Equation 8.6.24 using the Euler-Marugama method when h = 0.02, a/L 1, B/L 0, Io 0, and 
v(t) = R. The mean and 95% confidence interval (here tstudent = 2-228) are given by the heavy solid and 
dashed lines, respectively. Finally, the crosses (+) give the deterministic solution. 


8.6.21, a two-sided confidence interval can be determined as 


= a(tn) a(tn) 


X (tn) — Tstuden ~ Fe 
(tn) tudent TF Fi 


based on the student-7 distribution with J — 1 degrees of freedom. 


,X (tn) + Tstudent 


Project: RL Electrical Circuit with Noise 


An important component of contemporary modeling is the mixture of deterministic 
and stochastic aspects of a physical system. In this project you will see how this is done 
using a simple electrical system.!® 

Consider a simple electrical circuit consisting of a resistor with resistance R and an 
inductor with inductance L. If the circuit is driven by a voltage source u(t), the current 
I(t) at a given time t is given by the first-order ordinary differential equation 


dI 
L+R =v(t) (0) =. (8.6.21) 


Step 1: Using classical methods, show that the deterministic solution to Equation 8.6.21 is 


I(t) = he ®t + f EG = J oriin (8.6.22) 


16 See Kolářová, E., 2005: Modeling RL electrical circuits by stochastic differential equations. Proc. Int. 
Conf. Computers as Tool, Belgrade (Serbia and Montenegro), IEEE R8, 1236-1238. 
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Figure 8.6.3: Plot of E/x(t)] versus time for the FitzHugh-Nagamo model for three values of ø. The value 
of the parameters are a = 0.8, m = 1.2, and 7 = 100. The Euler method was used with a time step of 0.1. 


There are two possible ways that randomness can enter this problem. First, the power 
supply could introduce some randomness so that the right side of Equation 8.6.21 could 
read v(t) + adBo(t)/dt. Second, some physical process within the resistor could cause 
randomness so that the resistance would now equal R + 6 dB,(t)/dt. Here Bı (t) and Bo(t) 
denote two independent white noise processes and a, are nonnegative constants. In this 
case the governing differential equation would now read 


dI 1 dB, 1 dBo 

atz([Rte Z| 7 [+8 ls (0) = Do (8.6.23) 
Converting Equation 8.6.23 into the standard form of a stochastic ordinary differential 
equation, we have that 


dI = = [v(t) — RI(t)] dt — SIO) dB: (t) + A dBo(t), I0) =h. (8.6.24) 


Step 2: Using MATLAB, create a script to numerically integrate Equation 8.6.24 for a given 
set of a, 3, Io = 0, R, L, and v(t). Plot I(t) as a function of the nondimensional time Rt/L 
for many realizations (say 20). See Figure 8.6.2. 


Step 3: Although some idea of the effect of randomness is achieved by plotting several 
realizations, a better way would be to compute the mean and standard deviation at a given 
time. On the plot from the previous step, plot the mean and standard deviation of your 
solution as a function of nondimensional time. How does it compare to the deterministic 
solution? 


Project: Relaxation Oscillator with Brownian Motion Forcing 


The FitzHugh-Nagamo!” model describes excitable systems such as a neuron. We will 
modify it so that the forcing is due to Brown motion. The governing equations are 


dx = —z(x° — a”) dt — y dt + o dBy(t), 


17 FitzHugh, R., 1961: Impulses and physiological states in theoretical models of nerve membrane. 
Biophys. J., 1, 445-466; Nagumo, J., S. Arimoto, and S. Yoshizawa, 1962: An active pulse transmission 
line simulating nerve axon. Proc. IRE, 50, 2061-2070. 
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Figure 8.6.4: Plot of E[x(t)] versus time for the damped harmonic oscillator forced by Brownian motion. 
The value of the parameters are k = 1, y = 0.25, and alpha = At = 0.1. Five thousand realization were 
performed. 


and 
dy = (x — my) dt/T + o dBa(t), 


where a, m, o, and T are parameters. 

Write a MATLAB script to numerically integrate this modified Fitz Hugh-Nagamo model 
for various values of ø. Using many simulations, compute E[a(t)] as a function of time t. 
See Figure 8.6.3. What is the effect of the Brownian motion forcing? 


Project: Stochastically Damped Harmonic Oscillator 


The damped stochastic harmonic oscillator is governed by the stochastic differential 
equations: 


dv(t) = —yu(t) dt — k*x(t) dt — ax(t) dB(t), and dx(t) = v(t) dt, 


where k, a and y are real constants. This system of equations is of interest for two reasons: 
(1) The system is forced by Brownian motion. (2) The noise is multiplicative rather than 
additive because the forcing term is z(t) dB(t) rather than just dB(t). 

We could solve both equations numerically using Euler’s method.'® The purpose of 
this project is to introduce you to the Heun method. In the Heun method we first compute 
an estimate of the solution «* and v* by taking a Euler-like time step: 


g* =a,+u,At, and v* =v; —yuj,At— k?2,At— ax;,AB,, 


where x; and v; denote the displacement and velocity at time t; = iAt, At is the time step, 
and i = 0,1,2,.... With these estimates we compute the value for 2:4; and vi+ı using 


Ti+1 = Ti + 5 (v3 +v*)At, and v4. =U; - SV(vi +u")At jk’ (aj + 2*)At — ax; AB;. 


18 For further details, see Greiner, A., W. Strittmatter, and J. Honerkamp, 1988: Numerical integration 
of stochastic differential equations. J. Stat. Phys., 51, 95-108. 
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Figure 8.6.5: The mean time that it takes a particle to travel from X (0) = —1 to X = 0 in the double-well 
potential stated in the text. Sixty thousand realizations were used with a time step h. Two differential 
numerical schemes were used: the Euler-Marugama (crosses) and the Milstein (circles) methods. The curves 
are linear least-squares fits through the results. 


Qiang and Habib! developed a leapfrog algorithm to solve this problem. Because the 
algorithm is rather complicated, the interested student is referred to their paper. 

Write a MATLAB script to use the Euler and Huen methods to numerically integrate 
the stochastic harmonic oscillator when 10a = 4y = k = 1 and z(0) = v(0) = 0. Using 
many simulations, compute E[x(t)| as a function of time t. See Figure 8.6.4. What happens 
to the accuracy of the solution for larger values of At? 


Project: Mean First Passage Time 


The stochastic differential equation 


dX(t) = [X (t) — X? (t)] dt + dB(t) 


X2(t) +1 


describes the motion of a particle in a double-well potential V(x) = 24/4 — 27/2, subject 
to a spatially dependent random forcing when the acceleration X” (t) can be neglected. 
An important question is what is the average (mean) time that it takes a particle initially 
located at a minimum X(0) = —1 to reach the local maximum X(t) = 0. 

Write MATLAB code that computes X(t) as a function of time t. Using this code 
and creating N realizations, compute the length of time that it takes the particle to reach 
X(t) = 0 in each realization. Then compute the mean from those times and plot the results 
as a function Vh, the square root of the time step. See Figure 8.6.5. We used Vh rather 
than h following the suggestions of SeeBelberg and Petruccione.?? 


19 Qiang, J., and S. Habib, 2000: Second-order stochastic leapfrog algorithm for multiplicative noise 
Brownian motion. Phys. Review, 62, 7430-7437. 


20 SeeSelberg, M., and F. Petruccione, 1993: An improved algorithm for the estimation of the mean first 
passage of ordinary stochastic differential equations. Comput. Phys. Commun., 74, 247—255. 
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Figure 8.6.6: The probability of bankruptcy over a three-year period as a function of interest rate of a 
firm with initial wealth Xo = 500 and debt D = 100. Other parameters are h = 0.01 yr and u = 1.001/year. 
The units on ø is year™ 1/2. Five hundred thousand realizations were used to compute the probability. 


Project: Bankruptcy of a Company 
The stochastic differential equation?! 
dX(t) = [uX (t) — iD] dt + oX(t) dB(t), 0<t<T, 


with X(0) = Xo, describes the evolution with time t of the wealth X(t) of a firm. Here u 
and o denote the deterministic and stochastic evolution of the firm’s wealth, respectively, 
Xo is the initial wealth of the firm, and iD gives the amount of payment to a financier (bank) 
who initially loaned the firm the amount D at the interested rate i. Write a MATLAB code 
to simulate the wealth of a firm during its lifetime T given a known D, i and Xo with 
u = 1.001/year, and various values of ø. 

During the simulation there is a chance that the firm goes bankrupt at time t = T < 
T. This occurs when the stochastic process hits the barrier X(T) = 0. If n denotes the 
number of times that bankruptcy occurs in N simulations, the probability of bankruptcy 
is P[X(r) = 0] = n/N. Using your code for simulating a firm’s wealth, compute the 
probability of bankruptcy as a function of interest rate for a small (D = 20, Xo = 100), 
medium (D = 100, Xo = 500), and large (D = 200, Xo = 1000) firm. See Figure 8.6.6. 
How does the average value of 7 vary with interest rate? 


Further Readings 


Kloeden, P. E., and E. Platen, 1992: Numerical Solution of Stochastic Differential Equa- 
tions. Springer-Verlag, 632 pp. A solid book covering numerical schemes for solving stochas- 
tic differential equations. 


Mikosch, T., 1998: Elementary Stochastic Calculus with Finance in View. World Scientific, 
212 pp. Very well-crafted book on stochastic calculus. 


21 See Cerqueti, R., and A. G. Quaranta, 2012: The perspective of a bank in granting credit: An 
optimization model. Optim. Lett., 6, 867-882. 
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Euler’s formula, 2 
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using z-transform, 180 
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finite element, 285 
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Fourier transform, 78-91, 228-232 
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for low-frequency filter, 230 
for ordinary differential eqn, 223-243 
for wave equation, 247—256 


harmonic functions, complex, 15 
heat dissipation in disc brakes, 132-134 
heat equation, 129-153 
Hilbert pair, 196 
Hilbert transform, 195-215 
and convolution, 207-208 
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complex contour, 16-20 
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inverse 
Fourier transform, 78-87 
Hilbert transform, 196 
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by contour integration, 77-87 
inversion of Laplace transform 
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isolated singularities, 12 
It6 process, 386 
It’s integral, 406-411 
Itô’s lemma, 410-418 
Ito, Kiyhosi, 413 
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Laplace transform, 92-101 
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Laplace equation, 154-156 
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Laplace transform (contd.) 
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Laplace’s equation, 

solution by Laplace transforms, 154-156 
Laurent expansion, 29 
law of large numbers, 322 
line integral, 16-20 
linearity 
of Hilbert transform, 205 
of z-transform, 166 
low-frequency filter, 229-230 
low-pass filter, 390-392 


Markov chain 

state, 358 

state transition, 358 

time homogeneous, 358 
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method of partial fractions 

for Fourier transform, 77 

for z-transform, 175-177 
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Monte Carlo integration, 324 
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multivalued complex function, 8 


not simply connected, 20 
numerical solution 
of stochastic differential eqn, 427—434 


order of a pole, 30 


Parseval’s identity 

for z-transform, 180 
partial fraction expansion 

for z-transform, 175-177 
path in complex integrals, 17 
path independence in complex integrals, 22 
permutation, 301 
phase of the complex number, 2 
phasor amplitude, 213 
Poisson process, 377-382 

arrival time, 380 
polar form of a complex number, 2 
pole of order n, 30 
population growth and decay, 366-375 
positively oriented curve, 23 
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power spectrum, 354-357 

principal branch, 3 

probability 
Bernoulli distribution, 326 
Bernoulli trials, 310 
binomial distribution, 327 
characteristic function, 320 
combinations, 301 
conditional, 302 
continuous joint distribution, 335 
correlation, 341 
covariance, 339 
cumulative distribution, 310 
distribution function, 314 
event, 296 

elementary, 296 
simple, 296 

expectation, 319 
experiment, 295 
exponential distribution, 330 
Gaussian distribution, 331 
geometric distribution, 326 
independent events, 303-304 
joint probability mass function, 333 
law of total probability, 303 
marginal probability functions, 333 
mean, 319 
normal distribution, 331 
permutation, 301 
Poisson distribution, 328 
probability integral, 332 
probability mass function, 308 
random variable, 308 
sample point, 295 
sample space, 295 
standard normal distribution, 332 
uniform distribution, 329 
variance, 319 


quadrature phase shifting, 195 


radius of convergence, 27 
random differential equation, 387-388 
random process, 345-383 
autocorrelation function, 351 
Bernoulli process, 346 
Brownian motion, 395-403 
chemical kinetics, 370 
counting process, 347 
mean, 349 
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random process (contd.) 
power spectrum, 354 
realization, 345 
sample function, 345 
sample path, 345 
state, 345 
state space, 345 
variance, 349 
wide-sense stationary process, 352 
Wiener process, 402-403 
random variable, 308 
discrete, 308 
domain, 308 
identically distributed, 308 
independent, 308 
independent identically distributed, 308 
range, 308 
real definite integrals 
evaluation of, 37—43 
real part of a complex number, 1 
regular complex function, 9 
removable singularity, 30 
residue, 29 
residue theorem, 33-36 
Riemann, Georg Friedrich Bernhard, 12 
root locus method, 229 
roots of a complex number, 5-7 


Schouten-Van der Pol theorem, 99 
Schwarz-Christoffel transformation, 66-75 
set, 294 

complement, 294 

disjoint, 294 

element, 294 

empty, 294 

intersection, 294 

null, 294 

subset, 294 

union, 294 

universal, 294 
simple pole, 30 
simply closed curve, 20 
single side-band signal, 213 
single-valued complex function, 8 
singularity 

essential, 30 

isolated, 30 

pole of order n, 30 

removable, 30 


solution of ordinary differential equations 
by Fourier transform, 227-232 
steady-state transfer function, 227 
step response, 224-225 
stochastic calculus, 385-431 
Brownian motion, 396—403 
damped harmonic motion, 389 
derivative, 385 
differential equations, 419-423 
first-passage problem, 391-393 
integrating factor, 420 
Itô process, 385 
It’s integral, 406-411 
Itd’s lemma, 410-418 
low-pass filter, 390-392 
nonlinear oscillator, 431 
numerical solution, 427—430 
Euler-Marugama method, 427 
Milstein method, 428 
product rule, 416, 419 
random differential equations, 387-388 
RL electrical circuit with noise, 430 
wave motion due to random 
forcing, 394-396 
Wiener process, 403 
stochastic process, 345 
subtraction of complex numbers, 1 
superposition integral, 219 
for ordinary differential equations, 233 


Taylor expansion, 27 
telegraph equation, 106-115 
telegraph signal, 346, 353 
transfer function, 223 
transform 

Fourier, 78-91, 228-232 

Hilbert, 195-215 

Laplace, 92-101 

z-, 159-193 
transmission line, 106-115 
transmission probability matrix, 360 


variance, 318-320 
Venn diagram, 294 
Volterra equation of the second kind, 100 


wave equation, 100-127 

wave motion due to random forcing, 394-396 
Wiener process, 350, 403 

Wiener, Norbert, 401 


Index 


z-transform, 159-193 
basic properties of, 164-172 
convolution for, 170 
final-value theorem for, 168-169 
for solving difference equations, 183-188 
initial-value theorem for, 168 
inverse of, 173-181 
linearity of, 166 


multiplication by n, 169 

of a sequence multiplied by an 
exponential sequence, 166 

of a shifted sequence, 166-168 

of periodic sequences, 169—170 

their use in determining 
stability, 189-192 
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